Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade

# Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade
Source: [https://arxiv.org/html/2607.06503](https://arxiv.org/html/2607.06503)
###### Abstract

Large language model \(LLM\) agents solving multi\-step tasks frequently commit to trajectories that are doomed to fail, yet continue to consume substantial inference compute before the failure becomes observable\. We show that failure is predictable early from the agent’s*internal representations*: lightweight per\-round probes on hidden activations anticipate eventual episode failure as early as the first interaction round, where scorers reading only the agent’s observable behavior are barely better than chance\. We turn this signal into a practical*abort cascade*: one distribution\-free calibrated gate per round, with per\-round recall budgets jointly searched so that eventually\-successful episodes survive*all*gates at a user\-specified global rate; this episode\-level guarantee is the one that matters in deployment, since false\-abort risk accumulates across gates\. Across two agent models on TextCraft, the cascade meets every recall target from 90% to 97% and, at the 90% target, saves47\.1%±10\.3%47\.1\\%\\pm 10\.3\\%\(Qwen\-2\.5\-7B\) and37\.2%±8\.8%37\.2\\%\\pm 8\.8\\%\(Llama\-3\.2\-3B\) of inference compute,1\.61\.6–1\.7×1\.7\\timesthe best single\-gate policy\. An otherwise\-identical cascade reading only behavior saves roughly half as much, and adding behavioral features to the probe yields no further gain: the hidden states capture what behavior reveals\. Finally, we characterize the sample complexity of*certifying*high recall targets, telling practitioners which recall promises their data can, and provably cannot, back\. The code will be released soon\.

## Introduction

episode continues while every gate passes![Refer to caption](https://arxiv.org/html/2607.06503v1/x1.png)round11round22round33⋯\\cdotsy∈\{0,1\}y\\in\\\{0,1\\\}f1​\(h1\)\>τ1f\_\{1\}\(h\_\{1\}\)\>\\tau\_\{1\}?h1h\_\{1\}f2​\(h2\)\>τ2f\_\{2\}\(h\_\{2\}\)\>\\tau\_\{2\}?h2h\_\{2\}f3​\(h3\)\>τ3f\_\{3\}\(h\_\{3\}\)\>\\tau\_\{3\}?h3h\_\{3\}abort: stop episode, save remaining computeC−crC\-c\_\{r\}yesτr\\tau\_\{r\}: Clopper–Pearson\-calibrated so gaterrpasses≥tr\\geq t\_\{r\}of successes\(t1,…,tRg\)\(t\_\{1\},\\dots,t\_\{R\_\{g\}\}\): searched to maximize savingssubject to*global*recall≥ρ⋆\\geq\\rho^\{\\star\}Figure 1:Recall\-controlled abort cascade\. At each of the firstRgR\_\{g\}rounds, a linear probefrf\_\{r\}reads the agent’s hidden statehrh\_\{r\}and a gate aborts the episode if the failure score exceeds a calibrated thresholdτr\\tau\_\{r\}; an episode completes only if it survives every gate\. Thresholds carry per\-round distribution\-free recall guarantees, and the per\-round budgetstrt\_\{r\}are jointly searched so that the*episode\-level*success recall meets a user\-chosen targetρ⋆\\rho^\{\\star\}\.LLM\-based agents increasingly tackle long\-horizon tasks \(web navigation, tool use, embodied simulation\) in which a single episode spans many rounds of interaction and consumes thousands of generated tokens\. A large fraction of this compute is wasted: when an agent has misunderstood the task, entered an unrecoverable state, or begun to loop, the episode is already doomed long before it formally times out or returns a wrong answer\. If we could detect such doomed episodes early and abort them, the saved compute could be reallocated to retries, sampling additional trajectories, or simply reducing serving cost\.

Three obstacles stand in the way\. First, we need a signal that distinguishes doomed episodes from eventually\-successful ones*early*in the trajectory, and early is precisely when behavioral evidence is scarcest\. We find that scorers reading only the agent’s observable behavior are barely better than chance in the first round and become informative only around rounds 3–4, by which time over a third of episodes have already finished and much of the useful remaining compute is gone\. Lightweight probes on the agent’s internal activations show the opposite pattern: at the very first round they already match or exceed the surface scorer’s eventual peak, attained only two to three rounds later, and they reach their own peak at round 2 \(Figure[3](https://arxiv.org/html/2607.06503#Sx4.F3)\)\.

Second, any abort policy is only usable in deployment if it comes with a controllable guarantee on the harm it causes: aborting an episode that would have succeeded silently destroys task reward, so practitioners need to bound the rate of such false aborts*before*deployment, not observe it afterwards\. Third—and specific to the sequential nature of agent episodes, a monitor that re\-evaluates the episode at every round faces accumulating risk: even if each individual check rarely kills a good episode, a successful trajectory must survive*all*of them, so per\-round guarantees do not compose into the episode\-level guarantee that matters\.

We address these with a recall\-controlled*cascade*\(Figure[1](https://arxiv.org/html/2607.06503#Sx1.F1)\): at each of an episode’s firstRgR\_\{g\}rounds, a gate aborts the episode if its probe score exceeds a thresholdτr\\tau\_\{r\}\. Each threshold is calibrated so that an exact binomial \(Clopper–Pearson\) lower confidence bound on the gate’s survival rate for successful episodes meets a per\-round recall budgettrt\_\{r\}, and the budget vector\(t1,…,tRg\)\(t\_\{1\},\\dots,t\_\{R\_\{g\}\}\)is searched on a disjoint validation split to maximize compute savings subject to the*global*recall \(the fraction of eventually\-successful episodes that survive every gate\) meeting a user\-chosen target\. A safety margin makes the search robust to validation noise; a stricter variant replaces the margin with a certificate on the global recall itself, yielding a distribution\-free, a priori verifiable guarantee\.

The cascade is strongly compute\-positive\. At a 90% global recall target it saves47\.1%±10\.3%47\.1\\%\\pm 10\.3\\%of inference compute for a Qwen\-2\.5\-7B agent and37\.2%±8\.8%37\.2\\%\\pm 8\.8\\%for a Llama\-3\.2\-3B agent,1\.61\.6–1\.7×1\.7\\timesthe best*single\-gate*policy at the same target, confirming that the freedom to distribute recall budget across rounds, rather than spend it all at one decision point, is where much of the practical value lies\. At a conservative 95% target the cascade still roughly doubles the single\-gate savings, and its achieved recall matches its target in every configuration tested\. Swapping the probe for a behavior\-only scorer inside the identical cascade cuts savings roughly in half on the stronger agent, and stacking behavioral features onto the probe adds nothing, indicating the internal signal already captures the behavioral one\.

Finally, we give an honest account of what*certified*recall control costs in data\. Withnnsuccessful validation episodes, a one\-sided certificate at confidence95%95\\%can support recall targets only up to0\.051/n0\.05^\{1/n\}; at our scale this caps certifiable targets near0\.9740\.974, and indeed the certified variant abstains at targets0\.980\.98and0\.990\.99, which would require roughly1\.3×1\.3\\timesand2\.6×2\.6\\timesmore successful episodes than we have \(Figure[7](https://arxiv.org/html/2607.06503#Sx4.F7)\)\. We view this not as a weakness of the method but as its point: the same machinery that saves compute also tells the practitioner, before deployment, which promises the available data can and cannot back\.

Our contributions are:

- •We demonstrate across two agent models that eventual task failure of an LLM agent is predictable from internal activations within the first interaction rounds, whereas behavior\-only scorers become informative only at rounds 3–4, after most recoverable compute is spent\.
- •We propose the first abort policy for LLM agents that controls*episode\-level*success recall across multiple sequential decision points, via per\-round distribution\-free gates whose recall budgets are jointly optimized under a global constraint, with an optional certificate\.
- •We show the cascade saves up to47\.1%47\.1\\%of inference compute at 90% global recall \(1\.61\.6–1\.7×1\.7\\timesthe best single\-gate policy\) and characterize the data requirements for certifying stricter targets\.

## Method

Algorithm 1Recall\-Controlled Abort CascadeInput: labeled episodes𝒟\\mathcal\{D\}, global recall targetρ⋆\\rho^\{\\star\}, budget grid𝒯\\mathcal\{T\}, margin rule \(marginδ\\deltaor certificate levelαm\\alpha\_\{m\}\) Output: gates\{\(fr,τr\)\}r=1Rg\\\{\(f\_\{r\},\\tau\_\{r\}\)\\\}\_\{r=1\}^\{R\_\{g\}\}

1:Score all episodes by task\-grouped cross\-fitted probes

frf\_\{r\}
2:Partition tasks into calibration / validation / test

3:foreach candidate budget

𝐭∈𝒯\\mathbf\{t\}\\in\\mathcal\{T\}do

4:for

r=1,…,Rgr=1,\\dots,R\_\{g\}do

5:

τr​\(𝐭\)←\\tau\_\{r\}\(\\mathbf\{t\}\)\\leftarrowsmallest threshold whose Clopper–Pearson lower bound on per\-round survival of successful calibration episodes at

rris

≥tr\\geq t\_\{r\}
6:endfor

7:Simulate cascade on validation split; record global recall

ρ^val\\hat\{\\rho\}\_\{\\mathrm\{val\}\}and savings

8:endfor

9:

ℱ←\\mathcal\{F\}\\leftarrowcandidates passing the margin or certificate rule at

ρ⋆\\rho^\{\\star\}
10:if

ℱ=∅\\mathcal\{F\}=\\emptysetthen

11:Abstain \(no aborts\)

12:else

13:Deploy

𝐭⋆=arg⁡max𝐭∈ℱ\\mathbf\{t\}^\{\\star\}=\\arg\\max\_\{\\mathbf\{t\}\\in\\mathcal\{F\}\}validation savings

14:endif

15:Evaluate once on the test split

### Problem Setup

An agent episode is a sequence of interaction rounds\(s1,a1,s2,a2,…\)\(s\_\{1\},a\_\{1\},s\_\{2\},a\_\{2\},\\dots\)between an LLM policy and an environment, terminating with a binary outcomey∈\{0,1\}y\\in\\\{0,1\\\}\. We index rounds from one; episodes run up toRfull=20R\_\{\\mathrm\{full\}\}=20rounds\. Letcrc\_\{r\}denote the cumulative inference cost incurred through roundrrandCCthe total cost of the full episode\.

An*abort cascade*places a gate at each of the firstRgR\_\{g\}rounds \(r=1,…,Rgr=1,\\dots,R\_\{g\}; we useRg=6R\_\{g\}=6\)\. At gaterr, every episode still running is scored by a per\-round scorerfrf\_\{r\}and aborted iffr​\(x\)\>τrf\_\{r\}\(x\)\>\\tau\_\{r\}, losing any potential success but saving the remaining costC−crC\-c\_\{r\}\. Episodes terminate at different rounds; in our data, over a third of episodes finish within the first two rounds \(Figure[2](https://arxiv.org/html/2607.06503#Sx3.F2)\), so a gate at a late round sees only the episodes still alive there: late gates are cheap in recall but also guard less remaining compute\. This trade\-off is exactly what the cascade optimizes\.

Aborting an episode that would have succeeded is the critical error mode\. We target*global success recall*: among episodes withy=1y=1, the fraction that survive*every*gate and run to completion\. The design goal is to maximize expected cost savings subject to a user\-specified global recall floorρ⋆\\rho^\{\\star\}\. Note that controlling the recall of each gate in isolation does not control the global recall: false\-abort probability accumulates across gates, and the accumulation depends on how many successes each gate sees\.

### Per\-Round Failure Scorers

At each gate roundrr, we extract a feature vectorx∈ℝdx\\in\\mathbb\{R\}^\{d\}from the agent LLM’s internal activations: the residual\-stream hidden state at the final token of the agent’s generated action in roundrr, extracted via a teacher\-forced forward pass over the trajectory, at a single layer per model \(layer 14 of 28 for Llama\-3\.2\-3B, layer 20 of 28 for Qwen\-2\.5\-7B\)\. Layers were fixed once per model by preliminary per\-layer probe\-AUC sweeps: for Qwen\-2\.5\-7B over layers\{0,2,…,28\}\\\{0,2,\\dots,28\\\}on an independent pilot set of rollouts collected before the main experiments \(post\-generation AUC peaks at layer 20\), and for Llama\-3\.2\-3B over layers\{6,10,14,18,22,26\}\\\{6,10,14,18,22,26\\\}, where the post\-generation sweep selects layer 14 \(see the technical appendix, “Per\-Layer Probe AUC Sweeps”\)\. A per\-round probefrf\_\{r\}, a logistic regression on standardized features \(L2L\_\{2\}regularization,C=1C=1\), is trained to predict eventual failure1−y1\-yfrom episodes alive at roundrr\.

We compare against two alternative scorers, holding the entire downstream calibration and search pipeline fixed\. Thesurfacescorer observes only the agent’s*behavior*: a logistic model over trajectory features observable from the serving API alone: the mean action\-token log\-probability of the current round, the mean over preceding rounds, the number of generated tokens, the prefix length, and the count of preceding rounds whose environment feedback contains error keywords \(“error”, “invalid”, “fail”, etc\.\)\. Thestackingscorer concatenates these surface features onto the probe’s activation features, testing whether behavioral evidence adds anything beyond the hidden state\.

### Per\-Round Recall\-Calibrated Gates

Each gate’s threshold is set on a calibration split so that the gate provably passes at least atrt\_\{r\}fraction of successful episodes\. Given a per\-round recall budgettrt\_\{r\}, let𝒮r=\{fr​\(xi\):yi=1,i​alive at​r\}\\mathcal\{S\}\_\{r\}=\\\{f\_\{r\}\(x\_\{i\}\):y\_\{i\}=1,\\ i\\text\{ alive at \}r\\\}be the scores of thenrn\_\{r\}successful calibration episodes alive at roundrr\. For a candidate thresholdτ\\tau, letk​\(τ\)=\|\{s∈𝒮r:s≤τ\}\|k\(\\tau\)=\|\\\{s\\in\\mathcal\{S\}\_\{r\}:s\\leq\\tau\\\}\|be the number of survivors; the exact binomial \(Clopper–Pearson\) lower confidence bound on the gate’s true survival rate is the Beta quantile

p¯​\(τ\)=Beta−1​\(α;k​\(τ\),nr−k​\(τ\)\+1\),\\underline\{p\}\(\\tau\)\\;=\\;\\mathrm\{Beta\}^\{\-1\}\\\!\\left\(\\alpha;\\;k\(\\tau\),\\;n\_\{r\}\-k\(\\tau\)\+1\\right\),\(1\)and we setτr\\tau\_\{r\}to the smallest calibration score withp¯​\(τr\)≥tr\\underline\{p\}\(\\tau\_\{r\}\)\\geq t\_\{r\}, at per\-gate confidence level1−α1\-\\alphawithα=0\.05\\alpha=0\.05\. This is deliberately stronger than a marginal conformal quantile: the per\-round guarantee holds with high confidence over the calibration draw, at the price of conservatism whennrn\_\{r\}is small \(a gate whosenrn\_\{r\}cannot supporttrt\_\{r\}abstains, i\.e\., aborts nothing\)\. A conformal\-quantile variant \(τr=Quantile⌈\(nr\+1\)​tr⌉/nr​\(𝒮r\)\\tau\_\{r\}=\\mathrm\{Quantile\}\_\{\\lceil\(n\_\{r\}\+1\)t\_\{r\}\\rceil/n\_\{r\}\}\(\\mathcal\{S\}\_\{r\}\)\) is compared in the technical appendix \(“Conformal\-Quantile Gates”\)\. A budget oftr=1t\_\{r\}=1disables the gate\.

### Recall Budget Search Under a Global Constraint

Per\-round guarantees do not compose multiplicatively in any useful way: a union bound over gates is valid but hopelessly conservative, because it ignores that late gates expose few successes and that per\-gate false\-abort events are far from disjoint\. We therefore treat the budget vector𝐭=\(t1,…,tRg\)\\mathbf\{t\}=\(t\_\{1\},\\dots,t\_\{R\_\{g\}\}\)as a hyperparameter and select it empirically on a validation split disjoint from calibration\.

Concretely, probe scores are produced by task\-grouped cross\-fitting \(stratified groupkk\-fold\), so every episode is scored by a probe that never saw its task during training\. Tasks are then partitioned into a calibration set for gate thresholds \(20%20\\%of tasks\), a validation set for budget search \(20%20\\%\), and a held\-out test set \(60%60\\%\)\. Grouping by task throughout ensures no task contributes episodes to two sides of any split\.

For each candidate𝐭\\mathbf\{t\}on the gridtr∈\{0\.85,0\.90,0\.95,0\.98,0\.99,1\.0\}t\_\{r\}\\in\\\{0\.85,0\.90,0\.95,0\.98,0\.99,1\.0\\\}\(66=46,6566^\{6\}=46\{,\}656candidates\), we calibrate all gates on the calibration split, simulate the full cascade on the validation split, and record its global recallρ^val​\(𝐭\)\\hat\{\\rho\}\_\{\\mathrm\{val\}\}\(\\mathbf\{t\}\)and compute savings\. The deployed budget maximizes validation savings subject to a feasibility condition, for which we study two variants:

- •Margin\(default\): requireρ^val​\(𝐭\)≥ρ⋆\+δ\\hat\{\\rho\}\_\{\\mathrm\{val\}\}\(\\mathbf\{t\}\)\\geq\\rho^\{\\star\}\+\\deltawith a fixed safety marginδ=0\.02\\delta=0\.02\. This is a heuristic guard against selection bias from searching over many candidates; it carries no formal guarantee, and we validate it empirically\.
- •Certificate: require that the Clopper–Pearson lower bound at level1−αm1\-\\alpha\_\{m\}\(αm=0\.05\\alpha\_\{m\}=0\.05\) on the*global*recall, computed from thenposn\_\{\\mathrm\{pos\}\}successful validation episodes, exceedρ⋆\\rho^\{\\star\}\. This yields a distribution\-free guarantee that holds despite the search, at the price of conservatism analyzed below\.

If no candidate is feasible, the policy abstains and aborts nothing\. Two fixed budget allocations serve as structural baselines:single\-gate, which spends the entire recall budget at a single round, with both the round and its budget selected by the same validation search \(recovering the single\-decision\-point policies of prior monitoring work as a special case of our framework\), anduniform, which sets the sametrt\_\{r\}at every gate\.

#### Sample complexity of certification\.

The certificate makes visible a fundamental data requirement\. Even a candidate that aborts*nothing*has a validation lower bound ofBeta−1​\(αm;npos,1\)=αm1/npos\\mathrm\{Beta\}^\{\-1\}\(\\alpha\_\{m\};n\_\{\\mathrm\{pos\}\},1\)=\\alpha\_\{m\}^\{1/n\_\{\\mathrm\{pos\}\}\}\(the “rule of three”\), so targets above this level are unattainable regardless of scorer quality\. Our splits yieldnpos=113n\_\{\\mathrm\{pos\}\}=113\(Llama\) and115115\(Qwen\), capping certifiable targets near0\.9740\.974; the resulting sample complexity of stricter targets is analyzed in the Results \(Figure[7](https://arxiv.org/html/2607.06503#Sx4.F7)\) and extends directly when recalibrated on more data\.

## Experimental Setup

11223344556600\.50\.511Gate roundFraction of episodes aliveLlama\-3\.2\-3BQwen\-2\.5\-7BFigure 2:Fraction of episodes still running at each gate round\. Exposure decays sharply: over a third of episodes finish within two rounds, so late gates guard little remaining compute; this is the structural trade\-off the budget search exploits\.#### Environment and Agents\.

We use TextCraft\(Prasadet al\.[2024](https://arxiv.org/html/2607.06503#bib.bib51)\), a text\-based crafting environment from the AgentGym suite\(Xiet al\.[2024](https://arxiv.org/html/2607.06503#bib.bib54)\)in which an agent must synthesize a target item by navigating multi\-step crafting recipes, with binary task success\. We collect800800episodes over100100tasks for each of two agent policies: Llama\-3\.2\-3B\(Meta AI[2024](https://arxiv.org/html/2607.06503#bib.bib49)\)\(task success rate63\.0%63\.0\\%\) and Qwen\-2\.5\-7B\(Qwen Team[2025](https://arxiv.org/html/2607.06503#bib.bib50)\)\(74\.8%74\.8\\%\), sampling at temperature1\.01\.0\(8 episodes per task\)\. Episodes run up toRfull=20R\_\{\\mathrm\{full\}\}=20rounds; gates are placed at rounds11–66\. The population still alive at each gate decays quickly \(Figure[2](https://arxiv.org/html/2607.06503#Sx3.F2)\):35%35\\%\(Llama\) and39%39\\%\(Qwen\) of episodes already finish within two rounds, and53%53\\%/69%69\\%before the last gate, quantifying how sharply late gates’ exposure decays\. Compute savings are reported as the fraction of total generated tokens saved by aborts, counting only each aborted episode’s remaining computation\.

#### Protocol\.

Probes are per\-round logistic regressions \(C=1C=1, standardized features\) on layer\-14 \(Llama\) / layer\-20 \(Qwen\) activations\. The entire pipeline \(task\-level splitting, probe cross\-fitting, gate calibration, budget search, evaluation\) is repeated over2020random seeds; we report test\-split mean±\\pmstandard deviation\. All scorers and all budget\-allocation baselines share identical splits, calibration machinery, and search procedure, so differences isolate the quality of the underlying signal \(scorer comparisons\) or the value of distributing the recall budget \(allocation comparisons\)\.

## Results

### Internal States Predict Failure Before Behavior Does

Figure[3](https://arxiv.org/html/2607.06503#Sx4.F3)plots the cross\-fitted AUC of each per\-round scorer for predicting eventual failure among episodes alive at that round\. The behavior\-only surface scorer is barely better than chance at the first gate and becomes informative only around rounds 3–4, consistent with behavioral evidence \(repetitions, lack of progress, invalid actions\) needing time to accumulate\. The probe shows the opposite timing pattern: its*first\-round*AUC already matches or exceeds the surface scorer’s peak, which the latter attains only two to three rounds later, and the probe peaks at round 2, an absolute advantage of0\.120\.12–0\.210\.21AUC over the surface scorer in precisely the rounds where most of the episode’s compute is still unspent\. The two signals converge only at round 6, by which point the majority of episodes have already ended \(Figure[2](https://arxiv.org/html/2607.06503#Sx3.F2)\)\. The probe’s decline after round 2 reflects population filtering rather than fading signal: episodes that terminate early leave the alive\-set, concentrating the harder, longer\-running cases at later gates\. This timing difference matters more than peak values: discriminative power at rounds 1–2 protects far more useful remaining compute than the same power at rounds 3–4, a point the cascade results below make quantitative\.

1122334455660\.50\.50\.60\.60\.70\.70\.80\.80\.90\.9Gate roundAUCLlama\-3\.2\-3B112233445566Gate roundQwen\-2\.5\-7BProbeSurfaceFigure 3:Cross\-fitted AUC for predicting eventual episode failure at each gate round \(among episodes alive at that round\)\. Probes on internal activations peak at round 2; the behavior\-only surface scorer starts near chance and peaks only at rounds 3–4, after over a third of episodes have already finished\.
### Main Results: Cascade vs\. Single Gate

Table[1](https://arxiv.org/html/2607.06503#Sx4.T1)reports the compute savings of the abort cascade at four global recall targets for both agents, against the single\-gate and uniform budget allocations; Figure[4](https://arxiv.org/html/2607.06503#Sx4.F4)plots the recall each cascade actually achieves against its target, and Figure[5](https://arxiv.org/html/2607.06503#Sx4.F5)traces the savings–target frontier\. Three observations stand out\.

Table 1:Probe\-based abort cascade with Clopper–Pearson gates and margin\-based budget search \(δ=0\.02\\delta=0\.02; 20 seeds, test split\)\. Achieved global recall matches its target within one standard deviation in every row \(Figure[4](https://arxiv.org/html/2607.06503#Sx4.F4)\); the single\-gate and uniform baselines also meet their targets while staying on the conservative side\.0\.90\.90\.920\.920\.950\.950\.970\.970\.90\.90\.950\.9511Target global recallρ⋆\\rho^\{\\star\}Achieved test recallLlama\-3\.2\-3BQwen\-2\.5\-7BFigure 4:Achieved global success recall of the cascade vs\. its target \(mean±\\pmone standard deviation over 20 seeds; points horizontally jittered for legibility\)\. The diagonal marks exact targeting; every configuration lands on or above its target within one standard deviation\.First,*achieved recall matches its target in every configuration*\(Figure[4](https://arxiv.org/html/2607.06503#Sx4.F4)\): across all eight agent–target pairs the cascade’s mean test recall deviates from its target by at most0\.0270\.027, always within one standard deviation, and errs above the target in seven of eight cases\. The savings in Table[1](https://arxiv.org/html/2607.06503#Sx4.T1)are thus attained under honest risk control rather than post\-hoc threshold tuning\.

Second,*distributing the recall budget across rounds is worth1\.61\.6–2\.3×2\.3\\timesover the best single decision point*, at every target and for both agents \(Figure[5](https://arxiv.org/html/2607.06503#Sx4.F5)\); for Llama the ratio widens as the target tightens \(up to2\.3×2\.3\\timesat0\.970\.97\), while for Qwen it holds near1\.71\.7–1\.8×1\.8\\times\. The searched budgets explain why: the modal allocation at Llama’s0\.900\.90target is\(0\.95,0\.85,0\.85,1\.0,1\.0,1\.0\)\(0\.95,0\.85,0\.85,1\.0,1\.0,1\.0\): it spends recall aggressively at rounds 1–3, where doomed episodes are identifiable and most of their cost is still unspent, and disables late gates entirely\. The uniform allocation, forced to be equally strict at the recall\-expensive early gates and the low\-yield late ones, saves almost nothing, confirming that*where*the budget is spent, not just how much, drives the savings\.

Third,*the stronger agent yields larger savings*\. Qwen’s cascade saves more than Llama’s at every target despite Qwen’s higher success rate leaving fewer doomed episodes to abort\. Two factors contribute: Qwen’s failure signal is simply stronger at every gate \(Figure[3](https://arxiv.org/html/2607.06503#Sx4.F3)\), and its episodes finish earlier \(Figure[2](https://arxiv.org/html/2607.06503#Sx3.F2)\), concentrating surviving failures into an increasingly separable population at mid\-trajectory gates\.

0\.90\.90\.920\.920\.950\.950\.970\.97020204040Recall targetρ⋆\\rho^\{\\star\}Saved \(%\)Llama\-3\.2\-3B0\.90\.90\.920\.920\.950\.950\.970\.97Recall targetρ⋆\\rho^\{\\star\}Qwen\-2\.5\-7BCascadeSingle\-gateUniformFigure 5:Compute savings versus global recall target for the three budget allocations \(test split, mean over 20 seeds\)\. The cascade dominates at every target; the uniform allocation collapses at strict targets\.
### Does the Signal Require Internal Access?

Figure[6](https://arxiv.org/html/2607.06503#Sx4.F6)fixes the target at0\.950\.95and swaps the scorer inside the otherwise\-identical pipeline, for both the cascade and single\-gate allocations\. Three conclusions emerge\.

First,*internal access roughly doubles the savings of the strongest configuration*on Qwen \(32\.0%32\.0\\%for the probe cascade vs\.17\.0%17\.0\\%for the surface cascade\); on Llama the gap is present but narrower\. Second,*stacking behavioral features onto the probe adds nothing*: savings land within noise of the probe alone, indicating that whatever the surface scorer knows, the hidden states already encode; the converse is clearly false\. Third,*the cascade structure amplifies weak scorers even more than strong ones*: moving from single gate to cascade multiplies surface savings by roughly3\.6×3\.6\\times, versus about1\.9×1\.9\\timesfor the probe\. The per\-round AUCs of Figure[3](https://arxiv.org/html/2607.06503#Sx4.F3)explain this asymmetry: the surface signal only matures at rounds 3–4, so a single early gate cannot exploit it at all, whereas the cascade can defer part of the budget to the rounds where the behavioral evidence finally arrives\. Notably, the surface single gate remains weak at*every*round—even at its best placement it saves less than the probe single gate at round 1, so the cascade is a complement to, not a substitute for, a signal that arrives early\.

ProbeSurfaceStacking020204040Saved \(%\)Llama\-3\.2\-3BProbeSurfaceStackingQwen\-2\.5\-7BCascadeSingle\-gateFigure 6:Scorer ablation at target global recall0\.950\.95\(mean±\\pmone standard deviation over 20 seeds\)\. “Stacking” concatenates surface features onto the probe\. All configurations meet the recall target \(achieved recall0\.9540\.954–0\.9700\.970\)\.
### The Limits of Certification at High Recall

Replacing the empirical margin with the global certificate \(αm=0\.05\\alpha\_\{m\}=0\.05\) makes the guarantee formal, and its cost explicit\. Figure[7](https://arxiv.org/html/2607.06503#Sx4.F7)plots the minimum number of successful validation episodes required to certify a given target, the “rule of three” boundnpos≥ln⁡αm/ln⁡ρ⋆n\_\{\\mathrm\{pos\}\}\\geq\\ln\\alpha\_\{m\}/\\ln\\rho^\{\\star\}, against thenpos=113n\_\{\\mathrm\{pos\}\}=113/115115our splits provide\. Targets up to≈0\.974\\approx 0\.974lie below the curve and are certifiable;0\.980\.98and0\.990\.99lie above it, so*no*candidate, including the no\-op, can be certified, and the policy abstains with zero savings by construction\. This is not the scorer’s failure but the target’s data cost: even the*margin*\-based single gate at0\.980\.98\(with the margin in mean\-minus\-one\-σ\\sigmaform at these targets\) saves only1\.5%1\.5\\%–5\.5%5\.5\\%across the two agents, confirming that near\-perfect recall leaves little room for any abort policy at this data scale\. We regard this transparency as a feature: the method reports, before deployment, that800800episodes suffice to back a0\.950\.95–0\.970\.97promise but not a0\.990\.99one, and Figure[7](https://arxiv.org/html/2607.06503#Sx4.F7)prescribes exactly how much additional data the stricter promise costs\.

0\.90\.90\.920\.920\.940\.940\.960\.960\.980\.980100100200200300300our scale \(npos=113/115n\_\{\\mathrm\{pos\}\}=113/115\)149149299299Recall targetρ⋆\\rho^\{\\star\}Requirednposn\_\{\\mathrm\{pos\}\}ln⁡αm/ln⁡ρ⋆\\ln\\alpha\_\{m\}/\\ln\\rho^\{\\star\}Figure 7:Sample complexity of the global recall certificate \(αm=0\.05\\alpha\_\{m\}=0\.05\): minimum number of successful validation episodes below which even the no\-op policy cannot be certified\. Our splits \(dashed\) support targets up to≈0\.974\\approx 0\.974; targets of0\.980\.98and0\.990\.99\(dots\) require substantially more data, independent of scorer quality\.
### Does the Margin Actually Protect the Target?

The margin rule is a heuristic, so we test it directly\. Across the eight main configurations, the searched budget’s validation recall exceeds its test recall by0\.0080\.008–0\.0300\.030, precisely the selection optimism theδ=0\.02\\delta=0\.02margin is sized to absorb\. The margin absorbs it almost exactly and no more: in the worst case \(Llama at0\.920\.92\) the mean test recall lands0\.0010\.001under the nominal target, well within seed noise, while in all other configurations it lands above\. Figure[8](https://arxiv.org/html/2607.06503#Sx4.F8)visualizes this per seed\. Removing the margin entirely \(δ=0\\delta=0\) confirms it is load\-bearing: at the0\.950\.95target the same search lands at test recall0\.943±0\.0320\.943\\pm 0\.032\(Qwen\) and0\.942±0\.0240\.942\\pm 0\.024\(Llama\), both below target, while saving more \(41\.0%41\.0\\%/26\.3%26\.3\\%\), quantifying exactly the savings the margin trades for reliability\.

0\.90\.90\.910\.910\.920\.920\.930\.930\.940\.940\.950\.950\.960\.960\.970\.970\.980\.980\.990\.99110\.80\.80\.850\.850\.90\.90\.950\.9511Validation recallTest recallLlama\-3\.2\-3BQwen\-2\.5\-7BFigure 8:Validation vs\. test global recall of the searched budget, for all 20 seeds and four targets\. Points below the diagonal \(solid\) exhibit selection optimism; the dashed line marks theδ=0\.02\\delta=0\.02margin sized to absorb it\.

## Related Work

#### Predicting success from internal representations\.

Lightweight probes on frozen activations decode properties that a model does not reliably verbalize\(Alain and Bengio[2016](https://arxiv.org/html/2607.06503#bib.bib13); Belinkov[2022](https://arxiv.org/html/2607.06503#bib.bib14)\)\. For LLMs, such probes predict whether the model knows an answer\(Kadavathet al\.[2022](https://arxiv.org/html/2607.06503#bib.bib15)\), whether a statement is true\(Azaria and Mitchell[2023](https://arxiv.org/html/2607.06503#bib.bib16); Burnset al\.[2023](https://arxiv.org/html/2607.06503#bib.bib17); Marks and Tegmark[2024](https://arxiv.org/html/2607.06503#bib.bib18)\), and whether a generation is hallucinated\(Orgadet al\.[2025](https://arxiv.org/html/2607.06503#bib.bib19); Kossenet al\.[2024](https://arxiv.org/html/2607.06503#bib.bib20); Jiet al\.[2024](https://arxiv.org/html/2607.06503#bib.bib30)\), with the relevant signal repeatedly found to be*linearly*decodable\(Marks and Tegmark[2024](https://arxiv.org/html/2607.06503#bib.bib18); Burnset al\.[2023](https://arxiv.org/html/2607.06503#bib.bib17)\)\. Closer to our setting, internal states anticipate chain\-of\-thought success before completion\(Afzalet al\.[2025](https://arxiv.org/html/2607.06503#bib.bib22)\)and encode self\-verification signals in reasoning models\(Zhanget al\.[2025](https://arxiv.org/html/2607.06503#bib.bib21)\)\. For agents specifically,Padhiet al\.\([2026](https://arxiv.org/html/2607.06503#bib.bib29)\)recover linearly separable success/failure directions in the activations of LLM agents in ScienceWorld and AlfWorld, using conformal prediction*upstream*to label step\-level representations from sparse episode rewards, with representation steering as the downstream application\.Mehta \([2026](https://arxiv.org/html/2607.06503#bib.bib40)\)probe a related but distinct quantity:*representational commitment*, the early convergence of an agent’s hidden states across resampled continuations, and find it deliberately orthogonal to correctness, with committed\-correct and committed\-wrong runs indistinguishable, whereas our probes predict the outcome itself\. These results establish that outcome\-relevant signal exists in the residual stream and is linearly accessible; none of them ask when in an episode the signal becomes actionable, nor turn it into a stopping rule with a controllable error rate\. Both questions are the subject of this paper\.

#### Failure detection and monitoring of LLM agents\.

Failure analyses consistently find that decisive errors arise early and cascade through the remaining trajectory\(Zhuet al\.[2025](https://arxiv.org/html/2607.06503#bib.bib24); Liet al\.[2026](https://arxiv.org/html/2607.06503#bib.bib28)\), motivating online monitoring over post\-hoc attribution; AgentRx\(Barkeet al\.[2026](https://arxiv.org/html/2607.06503#bib.bib44)\)localizes the decisive failure step from completed execution trajectories, a source of step\-level supervision that per\-round probes like ours could in principle exploit\. Existing online monitors read trajectory text or derived trajectory embeddings: AgentForesight\(Zhanget al\.[2026](https://arxiv.org/html/2607.06503#bib.bib25)\)trains a separate 7B auditor with reinforcement learning to localize the earliest decisive error in multi\-agent systems,Baidyaet al\.\([2026](https://arxiv.org/html/2607.06503#bib.bib26)\)fit a weakly supervised text\-encoder alerter with a tunable accuracy–earliness trade\-off, and MASC\(Shenet al\.[2026](https://arxiv.org/html/2607.06503#bib.bib43)\)anomaly\-scores step embeddings against prototype\-guided next\-execution reconstructions to trigger self\-correction in multi\-agent systems\.Chen and others \([2026](https://arxiv.org/html/2607.06503#bib.bib45)\)compute inexpensive behavioral statistics over agent interactions, but for offline triage of logged trajectories rather than online intervention\. An alternative is to ask the agent itself: BAGEN\(Linet al\.[2026](https://arxiv.org/html/2607.06503#bib.bib27)\)formalizes budget awareness as verbalized interval prediction of remaining cost, but finds that frontier agents are systematically overoptimistic on doomed tasks and that even after SFT and RL the intervals reach at most 47% coverage, leaving calibration as an open problem; prompted early exit likewise trades success for cost without controlling the loss\(Luet al\.[2025](https://arxiv.org/html/2607.06503#bib.bib23)\)\. Our monitor differs on the two axes that matter for deployment: the signal is read from the agent’s own activations at negligible cost, requiring neither an auxiliary LLM pass per step nor trust in the agent’s self\-reports, and the abort decision carries an episode\-level, a priori verifiable bound on the rate of falsely terminated successes, which no existing monitor provides; existing alerters control at best a per\-check error rate, which degrades over the length of the episode\.

#### Conformal prediction and risk control for LLMs\.

Split conformal prediction and related distribution\-free calibration give finite\-sample guarantees under exchangeability\(Vovket al\.[2005](https://arxiv.org/html/2607.06503#bib.bib1); Papadopouloset al\.[2002](https://arxiv.org/html/2607.06503#bib.bib2); Leiet al\.[2018](https://arxiv.org/html/2607.06503#bib.bib3); Angelopoulos and Bates[2023](https://arxiv.org/html/2607.06503#bib.bib4)\), with extensions to general risks\(Bateset al\.[2021](https://arxiv.org/html/2607.06503#bib.bib5); Angelopouloset al\.[2025](https://arxiv.org/html/2607.06503#bib.bib6),[2024](https://arxiv.org/html/2607.06503#bib.bib7)\)\. For LLMs, these tools calibrate prediction sets and factuality filters\(Quachet al\.[2024](https://arxiv.org/html/2607.06503#bib.bib8); Mohri and Hashimoto[2024](https://arxiv.org/html/2607.06503#bib.bib9); Cherianet al\.[2024](https://arxiv.org/html/2607.06503#bib.bib10)\)and let planners ask for help with statistical guarantees on task completion\(Renet al\.[2023](https://arxiv.org/html/2607.06503#bib.bib11)\)\.Davidovet al\.\([2026](https://arxiv.org/html/2607.06503#bib.bib42)\)give a complementary formal treatment of when to quit, casting abstention as an explicit action in a regularized reinforcement\-learning objective and deriving value\-function conditions under which quitting is optimal; their guarantee is tied to the quality of a learned value estimate, whereas ours is finite\-sample and distribution\-free, and controls an episode\-level error rate across a*sequence*of gates\. Closest in spirit is CALM\(Schusteret al\.[2022](https://arxiv.org/html/2607.06503#bib.bib12)\), which calibrates per\-token early\-exit thresholds so that sequence\-level quality constraints hold with high probability; this is the same pattern of composing many local decisions under one global constraint that our cascade instantiates, at a different granularity and for a different risk: rather than skipping decoder layers under a text quality constraint, we terminate entire agent episodes under a constraint on global success recall\.

#### Adaptive allocation of inference compute\.

Reinforcement learning with verifiable rewards \(RLVR\) is an adjacent source of motivation: modern reasoning systems optimize sampled rollouts from task\-level correctness signals\. DeepSeekMath introduced Group Relative Policy Optimization \(GRPO\) for mathematical reasoning\(Shaoet al\.[2024](https://arxiv.org/html/2607.06503#bib.bib52)\), and DeepSeek\-R1 showed that large\-scale RL on verifiable tasks can elicit self\-reflection, verification, and strategy adaptation\(DeepSeek\-AI[2025](https://arxiv.org/html/2607.06503#bib.bib53)\)\. Our setting uses the same kind of episode\-level success signal, but at inference time: rather than updating the policy, we decide whether a running rollout should continue while preserving global success recall\.

A complementary literature adapts compute to instance difficulty: adaptive and early\-stopping self\-consistency truncate sampling once answers agree\(Aggarwalet al\.[2023](https://arxiv.org/html/2607.06503#bib.bib31); Liet al\.[2024](https://arxiv.org/html/2607.06503#bib.bib32); Manviet al\.[2024](https://arxiv.org/html/2607.06503#bib.bib33)\), cascades route queries from cheap to expensive models\(Chenet al\.[2023](https://arxiv.org/html/2607.06503#bib.bib34)\), optionally abstaining early when the large model is also likely to fail\(Zellingeret al\.[2025](https://arxiv.org/html/2607.06503#bib.bib35)\), and overthinking mitigation stops chain\-of\-thought once further reasoning is redundant\(Sunet al\.[2025](https://arxiv.org/html/2607.06503#bib.bib39); Maoet al\.[2025](https://arxiv.org/html/2607.06503#bib.bib46)\), terminating computation that has already*succeeded*, the mirror image of our problem\. Atropos\(Kim and Yoo[2026](https://arxiv.org/html/2607.06503#bib.bib41)\)is closest to ours in goal: a graph network over a semantic flow graph of sampled reasoning paths predicts whether an ongoing self\-consistency run will succeed, triggering early termination and a hotswap to a stronger model; the predictor reads generated text rather than internal states and provides no control on the rate of falsely terminated successes\. At the systems level, early\-exit architectures realize per\-token adaptive depth and expose the serving\-stack considerations any activation\-reading monitor inherits\(Chen and others[2023](https://arxiv.org/html/2607.06503#bib.bib47); Miaoet al\.[2024](https://arxiv.org/html/2607.06503#bib.bib48)\)\. In multi\-step settings, process reward models score partial trajectories to prune search branches\(Lightmanet al\.[2024](https://arxiv.org/html/2607.06503#bib.bib36); Zhanget al\.[2024](https://arxiv.org/html/2607.06503#bib.bib37); Xiaet al\.[2025](https://arxiv.org/html/2607.06503#bib.bib38)\), precisely because waiting for complete rollouts is costly\. These methods allocate compute across samples, models, or branches, generally without guarantees on the induced loss of task reward; we address the orthogonal single\-trajectory decision of whether a running episode is worth finishing at all, with recall control built in, so that the reported compute savings are honest by construction\.

## Discussion and Limitations

#### Margin vs\. certificate\.

Our default policy controls global recall through an empirically validated margin, not a theorem; the certified variant closes this gap but, at our data scale, only up to targets of≈0\.97\\approx 0\.97\. This mirrors the state of practice in selective prediction: formal episode\-level guarantees over sequential gates with searched budgets require either more calibration data or sharper multiple\-testing machinery \(e\.g\., fixed\-sequence or e\-value methods\) than the one\-shot bound we certify with\. We see tightening this loop as the main methodological open problem our results motivate\.

#### Search granularity and margin size\.

The budget grid is deliberately coarse\. A finer or continuous search could only improve validation savings, and its failure mode is conservative \(an infeasible or suboptimal grid point gives up savings, never recall\), so grid resolution trades efficiency, not safety\. The marginδ=0\.02\\delta=0\.02was fixed a priori and happens to match the binomial standard error of validation recall at our scale,ρ⋆​\(1−ρ⋆\)/npos≈0\.02\\sqrt\{\\rho^\{\\star\}\(1\-\\rho^\{\\star\}\)/n\_\{\\mathrm\{pos\}\}\}\\approx 0\.02forρ⋆=0\.95\\rho^\{\\star\}=0\.95andnpos≈114n\_\{\\mathrm\{pos\}\}\\approx 114; this gives a data\-driven recipe for sizingδ\\deltain a new domain, with the certificate available whenever a heuristic margin is unacceptable\.

#### Scope\.

Our evidence comes from one environment \(TextCraft\) with two agent models and800800episodes each; the exchangeability assumption underlying the gates can be violated under distribution shift \(e\.g\., new task types at deployment\)\. Recalibration on a small labeled sample restores the guarantee at modest cost\. Gates are placed at the first six rounds: beyond round 6 only a minority of episodes remain alive \(Figure[2](https://arxiv.org/html/2607.06503#Sx3.F2)\), per\-round AUC recedes toward its round\-1 level, and the searched budgets at strict targets already disable rounds 5–6, indicating little headroom further out\. Finally, our activations are extracted via an offline teacher\-forced re\-run; a deployed cascade would read them from the serving stack at generation time\. Since the feature is the activation at the final generated token of the round, no extra forward pass is required, but exposing intermediate activations inside optimized serving stacks \(fused attention kernels, paged KV caches\) is a real engineering task in its own right, as the early\-exit inference literature documents\(Chen and others[2023](https://arxiv.org/html/2607.06503#bib.bib47); Miaoet al\.[2024](https://arxiv.org/html/2607.06503#bib.bib48)\)\. Relatedly, we measure savings in generated tokens; realized wall\-clock or dollar savings depend additionally on batching, prefill–decode asymmetry, and whether the freed capacity is actually reused\.

#### What aborted compute buys\.

We report savings, not what to do with them\. The natural next step is closing the loop: reallocating aborted compute to retries of the same task turns recall\-controlled abort into a test\-time\-scaling policy whose reward effect can be measured end\-to\-end\.

## Conclusion

We showed that eventual failure of an LLM agent episode is predictable from internal activations within the first interaction rounds \(before behavior\-only monitors become informative\) across two agent models, and that a cascade of recall\-calibrated gates, with per\-round budgets searched under a global constraint, converts this signal into compute savings of up to47\.1%47\.1\\%at a90%90\\%global success\-recall target,1\.61\.6–1\.7×1\.7\\timeswhat any single decision point achieves\. The same framework tells practitioners which recall promises their data can certify and which it cannot\. Together these suggest a practical, honest route to cutting the inference cost of agentic LLM systems without silently sacrificing task success\.

## References

- A\. Afzal, F\. Matthes, G\. Chechik, and Y\. Ziser \(2025\)Knowing before saying: LLM representations encode information about chain\-of\-thought success before completion\.InFindings of the Association for Computational Linguistics: ACL,Cited by:[Predicting success from internal representations\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx1.p1.1)\.
- P\. Aggarwal, A\. Madaan, Y\. Yang, and Mausam \(2023\)Let’s sample step by step: adaptive\-consistency for efficient reasoning and coding with LLMs\.InEmpirical Methods in Natural Language Processing,Cited by:[Adaptive allocation of inference compute\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx4.p2.1)\.
- G\. Alain and Y\. Bengio \(2016\)Understanding intermediate layers using linear classifier probes\.arXiv preprint arXiv:1610\.01644\.Cited by:[Predicting success from internal representations\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx1.p1.1)\.
- A\. N\. Angelopoulos, S\. Bates, E\. J\. Candès, M\. I\. Jordan, and L\. Lei \(2025\)Learn then test: calibrating predictive algorithms to achieve risk control\.The Annals of Applied Statistics19\(2\),pp\. 1641–1662\.Cited by:[Conformal prediction and risk control for LLMs\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx3.p1.1)\.
- A\. N\. Angelopoulos, S\. Bates, A\. Fisch, L\. Lei, and T\. Schuster \(2024\)Conformal risk control\.InInternational Conference on Learning Representations,Cited by:[Conformal prediction and risk control for LLMs\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx3.p1.1)\.
- A\. N\. Angelopoulos and S\. Bates \(2023\)Conformal prediction: a gentle introduction\.Foundations and Trends in Machine Learning16\(4\),pp\. 494–591\.Cited by:[Conformal prediction and risk control for LLMs\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx3.p1.1)\.
- A\. Azaria and T\. Mitchell \(2023\)The internal state of an LLM knows when it’s lying\.InFindings of the Association for Computational Linguistics: EMNLP,Cited by:[Predicting success from internal representations\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx1.p1.1)\.
- A\. Baidya, X\. Liang, R\. Guo, X\. Gao, and K\. Das \(2026\)When evidence is sparse: weakly supervised early failure alerting in dialogs and LLM\-agent trajectories\.arXiv preprint arXiv:2606\.05414\.Cited by:[Failure detection and monitoring of LLM agents\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx2.p1.1)\.
- S\. Barke, A\. Goyal, A\. Khare, A\. Singh, S\. Nath, and C\. Bansal \(2026\)AgentRx: diagnosing AI agent failures from execution trajectories\.arXiv preprint arXiv:2602\.02475\.Cited by:[Failure detection and monitoring of LLM agents\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx2.p1.1)\.
- S\. Bates, A\. Angelopoulos, L\. Lei, J\. Malik, and M\. I\. Jordan \(2021\)Distribution\-free, risk\-controlling prediction sets\.Journal of the ACM68\(6\),pp\. 1–34\.Cited by:[Conformal prediction and risk control for LLMs\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx3.p1.1)\.
- Y\. Belinkov \(2022\)Probing classifiers: promises, shortcomings, and advances\.Computational Linguistics48\(1\),pp\. 207–219\.Cited by:[Predicting success from internal representations\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx1.p1.1)\.
- C\. Burns, H\. Ye, D\. Klein, and J\. Steinhardt \(2023\)Discovering latent knowledge in language models without supervision\.InInternational Conference on Learning Representations,Cited by:[Predicting success from internal representations\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx1.p1.1)\.
- L\. Chen, M\. Zaharia, and J\. Zou \(2023\)FrugalGPT: how to use large language models while reducing cost and improving performance\.arXiv preprint arXiv:2305\.05176\.Cited by:[Adaptive allocation of inference compute\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx4.p2.1)\.
- S\. Chenet al\.\(2026\)Signals: trajectory sampling and triage for agentic interactions\.arXiv preprint arXiv:2604\.00356\.Cited by:[Failure detection and monitoring of LLM agents\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx2.p1.1)\.
- Y\. Chenet al\.\(2023\)EE\-LLM: large\-scale training and inference of early\-exit large language models with 3D parallelism\.arXiv preprint arXiv:2312\.04916\.Cited by:[Adaptive allocation of inference compute\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx4.p2.1),[Scope\.](https://arxiv.org/html/2607.06503#Sx6.SSx5.SSSx3.p1.1)\.
- J\. J\. Cherian, I\. Gibbs, and E\. J\. Candès \(2024\)Large language model validity via enhanced conformal prediction methods\.InAdvances in Neural Information Processing Systems,Cited by:[Conformal prediction and risk control for LLMs\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx3.p1.1)\.
- H\. Davidov, N\. Cohen, O\. Kalinsky, Y\. Fairstein, G\. Kushilevitz, R\. Yazdi, and P\. Rebeschini \(2026\)Knowing when to quit: a principled framework for dynamic abstention in LLM reasoning\.arXiv preprint arXiv:2604\.18419\.Cited by:[Conformal prediction and risk control for LLMs\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx3.p1.1)\.
- DeepSeek\-AI \(2025\)DeepSeek\-R1: incentivizing reasoning capability in LLMs via reinforcement learning\.Nature645,pp\. 633–638\.Cited by:[Adaptive allocation of inference compute\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx4.p1.1)\.
- Z\. Ji, D\. Chen, E\. Ishii, S\. Cahyawijaya, Y\. Bang, B\. Wilie, and P\. Fung \(2024\)LLM internal states reveal hallucination risk faced with a query\.InProceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP,Cited by:[Predicting success from internal representations\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx1.p1.1)\.
- S\. Kadavath, T\. Conerly, A\. Askell, T\. Henighan, D\. Drain,et al\.\(2022\)Language models \(mostly\) know what they know\.arXiv preprint arXiv:2207\.05221\.Cited by:[Predicting success from internal representations\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx1.p1.1)\.
- N\. Kim and S\. Yoo \(2026\)Atropos: improving cost\-benefit trade\-off of LLM\-based agents under self\-consistency with early termination and model hotswap\.arXiv preprint arXiv:2604\.15075\.Cited by:[Adaptive allocation of inference compute\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx4.p2.1)\.
- J\. Kossen, J\. Han, M\. Razzak, L\. Schut, S\. Malik, and Y\. Gal \(2024\)Semantic entropy probes: robust and cheap hallucination detection in LLMs\.arXiv preprint arXiv:2406\.15927\.Cited by:[Predicting success from internal representations\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx1.p1.1)\.
- J\. Lei, M\. G’Sell, A\. Rinaldo, R\. J\. Tibshirani, and L\. Wasserman \(2018\)Distribution\-free predictive inference for regression\.Journal of the American Statistical Association113\(523\),pp\. 1094–1111\.Cited by:[Conformal prediction and risk control for LLMs\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx3.p1.1)\.
- X\. Li, W\. Yan, Y\. Wu, P\. Liang, M\. Yuan, J\. Liu, and J\. Yang \(2026\)Early diagnosis of wasted computation in multi\-agent LLM systems via failure\-aware observability\.arXiv preprint arXiv:2606\.01365\.Cited by:[Failure detection and monitoring of LLM agents\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx2.p1.1)\.
- Y\. Li, P\. Yuan, S\. Feng, B\. Pan, X\. Wang, B\. Sun, H\. Wang, and K\. Li \(2024\)Escape sky\-high cost: early\-stopping self\-consistency for multi\-step reasoning\.InInternational Conference on Learning Representations,Cited by:[Adaptive allocation of inference compute\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx4.p2.1)\.
- H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. Cobbe \(2024\)Let’s verify step by step\.InInternational Conference on Learning Representations,Cited by:[Adaptive allocation of inference compute\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx4.p2.1)\.
- Y\. Lin, Z\. Wang, M\. Liu, Y\. Shan, L\. Bai, J\. Zhang, X\. Jin, B\. Chen, J\. Su, X\. Wang, J\. Pei, and M\. Li \(2026\)BAGEN: are LLM agents budget\-aware?\.arXiv preprint arXiv:2606\.00198\.Cited by:[Failure detection and monitoring of LLM agents\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx2.p1.1)\.
- Q\. Lu, L\. Ding, S\. Cao, X\. Liu, K\. Zhang, J\. Zhang, and D\. Tao \(2025\)Runaway is ashamed, but helpful: on the early\-exit behavior of large language model\-based agents in embodied environments\.InFindings of the Association for Computational Linguistics: EMNLP,Cited by:[Failure detection and monitoring of LLM agents\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx2.p1.1)\.
- R\. Manvi, A\. Singh, and S\. Ermon \(2024\)Adaptive inference\-time compute: LLMs can predict if they can do better, even mid\-generation\.arXiv preprint arXiv:2410\.02725\.Cited by:[Adaptive allocation of inference compute\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx4.p2.1)\.
- M\. Mao, B\. Yin, Y\. Zhu, and X\. Fang \(2025\)Early stopping chain\-of\-thoughts in large language models\.arXiv preprint arXiv:2509\.14004\.Cited by:[Adaptive allocation of inference compute\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx4.p2.1)\.
- S\. Marks and M\. Tegmark \(2024\)The geometry of truth: emergent linear structure in large language model representations of true/false datasets\.InConference on Language Modeling,Cited by:[Predicting success from internal representations\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx1.p1.1)\.
- A\. Mehta \(2026\)When agents commit too soon: diagnosing premature commitment in LLM agents\.arXiv preprint arXiv:2606\.22936\.Cited by:[Predicting success from internal representations\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx1.p1.1)\.
- Meta AI \(2024\)Llama 3\.2: revolutionizing edge AI and vision with open, customizable models\.Note:https://ai\.meta\.com/blog/llama\-3\-2\-connect\-2024\-vision\-edge\-mobile\-devices/Cited by:[Environment and Agents\.](https://arxiv.org/html/2607.06503#Sx3.SSx4.SSSx1.p1.12)\.
- R\. Miao, Y\. Yan, X\. Yao, and T\. Yang \(2024\)An efficient inference framework for early\-exit large language models\.arXiv preprint arXiv:2407\.20272\.Cited by:[Adaptive allocation of inference compute\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx4.p2.1),[Scope\.](https://arxiv.org/html/2607.06503#Sx6.SSx5.SSSx3.p1.1)\.
- C\. Mohri and T\. Hashimoto \(2024\)Language models with conformal factuality guarantees\.InInternational Conference on Machine Learning,Cited by:[Conformal prediction and risk control for LLMs\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx3.p1.1)\.
- H\. Orgad, M\. Toker, Z\. Gekhman, R\. Reichart, I\. Szpektor, H\. Kotek, and Y\. Belinkov \(2025\)LLMs know more than they show: on the intrinsic representation of LLM hallucinations\.InInternational Conference on Learning Representations,Cited by:[Predicting success from internal representations\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx1.p1.1)\.
- T\. Padhi, R\. Kaur, K\. Agarwal, A\. D\. Cobb, D\. Elenius, M\. Acharya, C\. Samplawski, A\. M\. Berenbeim, N\. D\. Bastian, S\. Jha, and A\. Roy \(2026\)From actions to understanding: conformal interpretability of temporal concepts in LLM agents\.arXiv preprint arXiv:2604\.19775\.Cited by:[Predicting success from internal representations\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx1.p1.1)\.
- H\. Papadopoulos, K\. Proedrou, V\. Vovk, and A\. Gammerman \(2002\)Inductive confidence machines for regression\.InEuropean Conference on Machine Learning,Cited by:[Conformal prediction and risk control for LLMs\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx3.p1.1)\.
- A\. Prasad, A\. Koller, M\. Hartmann, P\. Clark, A\. Sabharwal, M\. Bansal, and T\. Khot \(2024\)ADaPT: as\-needed decomposition and planning with language models\.InFindings of the Association for Computational Linguistics: NAACL 2024,Cited by:[Environment and Agents\.](https://arxiv.org/html/2607.06503#Sx3.SSx4.SSSx1.p1.12)\.
- V\. Quach, A\. Fisch, T\. Schuster, A\. Yala, J\. H\. Sohn, T\. S\. Jaakkola, and R\. Barzilay \(2024\)Conformal language modeling\.InInternational Conference on Learning Representations,Cited by:[Conformal prediction and risk control for LLMs\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx3.p1.1)\.
- Qwen Team \(2025\)Qwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.Cited by:[Environment and Agents\.](https://arxiv.org/html/2607.06503#Sx3.SSx4.SSSx1.p1.12)\.
- A\. Z\. Ren, A\. Dixit, A\. Bodrova, S\. Singh, S\. Tu, N\. Brown, P\. Xu, L\. Takayama, F\. Xia, J\. Varley, Z\. Xu, D\. Sadigh, A\. Zeng, and A\. Majumdar \(2023\)Robots that ask for help: uncertainty alignment for large language model planners\.InConference on Robot Learning,Cited by:[Conformal prediction and risk control for LLMs\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx3.p1.1)\.
- T\. Schuster, A\. Fisch, J\. Gupta, M\. Dehghani, D\. Bahri, V\. Q\. Tran, Y\. Tay, and D\. Metzler \(2022\)Confident adaptive language modeling\.InAdvances in Neural Information Processing Systems,Cited by:[Conformal prediction and risk control for LLMs\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx3.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo \(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[Adaptive allocation of inference compute\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx4.p1.1)\.
- X\. Shen, Q\. Zhang, S\. Wang, Z\. Tan, X\. Zhao, L\. Yao, V\. Tadiparthi, H\. N\. Mahjoub, E\. Moradi Pari, K\. Lee, and T\. Chen \(2026\)Metacognitive self\-correction for multi\-agent system via prototype\-guided next\-execution reconstruction\.InFindings of the Association for Computational Linguistics: ACL 2026,Cited by:[Failure detection and monitoring of LLM agents\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx2.p1.1)\.
- R\. Sun, W\. Cheng, D\. Li, H\. Chen, and W\. Wang \(2025\)Stop when enough: adaptive early\-stopping for chain\-of\-thought reasoning\.arXiv preprint arXiv:2510\.10103\.Cited by:[Adaptive allocation of inference compute\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx4.p2.1)\.
- V\. Vovk, A\. Gammerman, and G\. Shafer \(2005\)Algorithmic learning in a random world\.Springer\.Cited by:[Conformal prediction and risk control for LLMs\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx3.p1.1)\.
- Z\. Xi, Y\. Ding, W\. Chen, B\. Hong, H\. Guo, J\. Wang, D\. Yang, C\. Liao, X\. Guo, W\. He, S\. Gao, L\. Chen, R\. Zheng, Y\. Zou, T\. Gui, Q\. Zhang, X\. Qiu, X\. Huang, Z\. Wu, and Y\. Jiang \(2024\)AgentGym: evolving large language model\-based agents across diverse environments\.arXiv preprint arXiv:2406\.04151\.Cited by:[Environment and Agents\.](https://arxiv.org/html/2607.06503#Sx3.SSx4.SSSx1.p1.12)\.
- Y\. Xia, J\. Fan, W\. Chen, S\. Yan, X\. Cong, Z\. Zhang, Y\. Lu, Y\. Lin, Z\. Liu, and M\. Sun \(2025\)AgentRM: enhancing agent generalization with reward modeling\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Cited by:[Adaptive allocation of inference compute\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx4.p2.1)\.
- M\. J\. Zellinger, R\. Liu, and M\. Thomson \(2025\)Cost\-saving LLM cascades with early abstention\.arXiv preprint arXiv:2502\.09054\.Cited by:[Adaptive allocation of inference compute\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx4.p2.1)\.
- A\. Zhang, Y\. Chen, J\. Pan, C\. Zhao, A\. Panda, J\. Li, and H\. He \(2025\)Reasoning models know when they’re right: probing hidden states for self\-verification\.arXiv preprint arXiv:2504\.05419\.Cited by:[Predicting success from internal representations\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx1.p1.1)\.
- B\. Zhang, J\. Zhu, Z\. Shi, D\. Liu, and R\. Tang \(2026\)AgentForesight: online auditing for early failure prediction in multi\-agent systems\.arXiv preprint arXiv:2605\.08715\.Cited by:[Failure detection and monitoring of LLM agents\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx2.p1.1)\.
- D\. Zhang, S\. Zhoubian, Z\. Hu, Y\. Yue, Y\. Dong, and J\. Tang \(2024\)ReST\-MCTS\*: LLM self\-training via process reward guided tree search\.InAdvances in Neural Information Processing Systems,Cited by:[Adaptive allocation of inference compute\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx4.p2.1)\.
- K\. Zhu, Z\. Liu, B\. Li, M\. Tian, Y\. Yang, J\. Zhang, P\. Han, Q\. Xie, F\. Cui, W\. Zhang, X\. Ma, X\. Yu, G\. Ramesh, J\. Wu, Z\. Liu, P\. Lu, J\. Zou, and J\. You \(2025\)Where LLM agents fail and how they can learn from failures\.arXiv preprint arXiv:2509\.25370\.Cited by:[Failure detection and monitoring of LLM agents\.](https://arxiv.org/html/2607.06503#Sx5.SSx5.SSSx2.p1.1)\.

Appendix

### A\. Per\-Layer Probe AUC Sweeps

Probe layers were fixed once per model and used unchanged in all experiments\. For Qwen\-2\.5\-7B, a per\-layer sweep of the same logistic probe over layers\{0,2,…,28\}\\\{0,2,\\dots,28\\\}was run on an independent pilot set of rollouts collected before the main experiments; the post\-generation AUC peaks at layer 20\. For Llama\-3\.2\-3B, the sweep covered layers\{6,10,14,18,22,26\}\\\{6,10,14,18,22,26\\\}; the post\-generation sweep selects layer 14\.

04488121216162020242428280\.60\.60\.70\.70\.80\.8AUCQwen\-2\.5\-7B \(pilot set\)post\-gen66101014141818222226260\.70\.70\.750\.750\.80\.8LayerAUCLlama\-3\.2\-3Bpost\-genFigure 9:Per\-layer probe\-AUC sweeps used to fix the probe layer for each agent\. Qwen\-2\.5\-7B: independent pilot set, layer 20\. Llama\-3\.2\-3B: post\-generation sweep, layer 14\.
### B\. Conformal\-Quantile Gates

Table[2](https://arxiv.org/html/2607.06503#A0.T2)compares the Clopper–Pearson gate of the main text against the conformal\-quantile variant, for a single probe\-based gate at round 1 with per\-round target0\.950\.95\(20 seeds\)\. The quantile gate controls recall only on average over the calibration draw, so with the small per\-round calibration sets available here its realized recall fluctuates below target, violating it for Llama \(0\.9330\.933vs\.0\.950\.95\) while saving more compute\. The Clopper–Pearson gate is conservative \(recall0\.9720\.972–0\.9770\.977at target0\.950\.95\) but never violates\. This is the trade the main text accepts: the cascade’s budget search recovers much of the conservatism by spending recall where it is cheap, while retaining a high\-confidence per\-gate guarantee\.

Table 2:Single gate at round 1, per\-round recall target0\.950\.95, probe scorer, 20 seeds\. Quantile calibration saves more but violates the target on Llama; Clopper–Pearson is conservative and always meets it\.