Cool paper from PwC. “Earlier is always better” is the default intuition for agent clarification. New paper claims that’s mostly wrong. Goal clarification loses nearly all of its value after just 10% of execution. The team built a forced-injection framework that drops ground-truth clarifications at controlled points along a long-horizon agent’s trajectory, across 4 information dimensions (goal, input, constraint, context), 3 benchmarks, and 4 frontier models. 84 task variants, 6,000+ runs. Pass@3 falls from 0.78 back to baseline. Input clarification keeps value through roughly 50%. Past mid-trajectory, asking any clarification at all performs worse than never asking. A complementary study of 300 unscripted sessions shows no current frontier model asks within the empirically optimal window. 52% of sessions over-ask. Others never ask at all. Why it matters: clarification has been treated as a binary capability, does the agent ask or not. This is the first quantitative demand curve for when the question is worth asking. Paper: https://arxiv.org/abs/2605.07937 Learn to build effective AI agents in our academy: https://academy.dair.ai

Ask Early, Ask Late, Ask Right: When Does Clarification Timing Matter for Long-Horizon Agents?

Source: https://arxiv.org/html/2605.07937 Anmol Gulati Hariom Gupta Elias Lumer Sahil Sen Vamse Kumar Subbiah PricewaterhouseCoopers U.S

Abstract

Long-horizon AI agents execute complex workflows spanning hundreds of sequential actions, yet a single wrong assumption early on can cascade into irreversible errors. When instructions are incomplete, the agent must decide not onlywhetherto ask for clarification butwhen, and no prior work measures how clarification value changes over the course of execution. We introduce a forced-injection framework that provides ground-truth clarifications at controlled points in the agent’s trajectory across four information dimensions (goal, input, constraint, context), three agent benchmarks, and four frontier models (three per benchmark; one on a single benchmark only; 84 task variants; 6,000+ runs). Counter to the common intuition that “earlier is always better,” we find that the value of clarification depends sharply onwhatinformation is missing: goal clarification loses nearly all value after 10% of execution (pass@3 drops from 0.78 to baseline), while input clarification retains value through roughly 50%. Deferringanyclarification type past mid-trajectory degrades performance below never asking at all. Cross-model Kendallτ\taucorrelations (0.78–0.87 among models sharing identical task coverage; 0.34–0.67 across the full 4-model panel) confirm these timing profiles are substantially task-intrinsic. A complementary study of 300 unscripted sessions reveals that no current frontier model asks within the empirically optimal window, with strategies ranging from over-asking (52% of sessions) to never asking at all. These empirical demand curves provide the quantitative foundation that existing theoretical frameworks require but have lacked, and establish concrete design targets for timing-aware clarification policies. Code and data will be publicly released.

1Introduction

Long-horizon AI agents now execute complex multi-step workflows spanning code repair, enterprise automation, and tool orchestration, often completing tens to hundreds of sequential actions with minimal human oversight(Yaoet al.,2023; Xuet al.,2024; Jimenezet al.,2024; Bandiet al.,2026; Lumeret al.,2026a). When user instructions are incomplete, these agents face a fundamental decision: continue executing under uncertainty, or pause to request clarification. Consider an enterprise agent tasked with generating a quarterly report that spends 30 actions building the wrong table before realizing the user meant fiscal quarters, not calendar quarters. A single clarifying question at step two could have prevented the wasted work entirely. The cost of this decision scales with trajectory length, as each additional action taken under wrong assumptions compounds the risk of irreversible errors and wasted computation(Lumeret al.,2026b).

Recent benchmarks show that underspecification consistently degrades agent performance and that clarification channels enable partial recovery(Puet al.,2026; Elfekiet al.,2026), yet existing work treats clarification as a binary capability: the agent either can or cannot ask. Neither LHAW(Puet al.,2026)nor HiL-Bench(Elfekiet al.,2026)addresses the temporal dimension, though LHAW’s failure mode analysis independently identifies timing as a problem, finding that agents ask at inappropriate points in the trajectory. Classical value-of-information theory(Howard,1966)and HCI interruption research(Adamczyk and Bailey,2004; Market al.,2008)both predict that clarification benefit should vary with timing, yet no empirical measurement of this interaction exists for modern LLM agents. Asking too early risks querying about information that is inferable from context not yet observed. Asking too late means the agent has already committed irreversible actions based on wrong assumptions.

We isolate the pure timing effect with a forced-injection framework that provides ground-truth clarifications at controlled trajectory points, independent of the agent’s ability to detect ambiguity, across 84 underspecified task variants, three benchmarks, and four frontier models (Figure1). Our experiments yield two findings that challenge common assumptions about clarification timing. First, the value of clarification is not simply “earlier is better”; it depends sharply on what type of information is missing, with goal clarification losing nearly all value after 10% of execution while input clarification retains value through roughly 50%. Second, despite this structure, no current frontier model naturally asks within the empirically optimal window for any dimension, with strategies ranging from over-asking (52% of sessions) to never asking at all.

Our contributions are:

1. 1.We establish that clarification timing interacts strongly with information dimension: goal information has a narrow early window (≤{\leq}10% of execution), input information is recoverable through∼{\sim}50%, and constraint benefit depends on whether an oracle gap exists, with cross-model consistency (Kendallτ\tauof 0.78–0.87 among models sharing identical task coverage; 0.34–0.67 on the broader panel) confirming these profiles are substantially task-intrinsic. Wasted compute grows steadily with delay, providing a complementary cost measure (Section5).
2. 2.We quantify the gap between current model asking behavior and these empirically optimal windows: across 300 unscripted sessions, natural ask timings are severely late for goal tasks, and one frontier model never asks at all, establishing concrete design targets for timing-aware clarification policies.

Figure 1:Overview of the forced-injection experimental framework. We inject ground-truth clarifications at controlled points along an oracle-calibrated action budget, measuring task success (pass@3) at each injection timing across four information dimensions.

2Related Work

Prior work addresses whether clarification helps, when information is theoretically valuable, and how agents should decide to ask. We identify a common blind spot: none varies clarificationtimingas an independent variable.

2.1Clarification in Agent Benchmarks

A growing body of work establishes that underspecification degrades agent performance and that clarification channels enable partial recovery. LHAW(Puet al.,2026)constructs 285 underspecified task variants across four information dimensions and shows that models exhibit widely divergent clarification strategies. HiL-Bench(Elfekiet al.,2026)complements this by penalizing both over-asking and missed escalation. Shorter-horizon benchmarks study related phenomena: QuestBench(Liet al.,2025), AmbigQA(Minet al.,2020), ClariQ(Aliannejadiet al.,2020), UserBench(Qian and others,2025), andτ\tau-bench(Yao and others,2024). Tamkin et al.(Tamkinet al.,2023)show that task ambiguity systematically degrades model behavior, and Kadavath et al.(Kadavathet al.,2022)establish that LLMs have limited self-assessment of uncertainty. Several approaches aim to close this gap: confidence-based selective clarification(Kuhnet al.,2023), uncertainty-aware planning(Huet al.,2024), proactive pre-execution questioning(Zhanget al.,2024), and simulation-based ask-or-answer training(Wuet al.,2025). In reinforcement learning, ask-for-help policies(Clouse,1996)and DAgger(Rosset al.,2011)query experts under uncertainty, Plaut et al.(Plautet al.,2025)prove that agents must query at a linear rate or risk catastrophe, and recent LLM systems learnwhetherto ask via counterfactual training(Liu and others,2025), process reward models(Minet al.,2025), or interactive feedback(Pan and others,2025). All optimize a confidence threshold but assume information is equally useful at any trajectory point. Broader long-horizon agent benchmarks(Zhouet al.,2024; Trivediet al.,2024; Liuet al.,2024; Xieet al.,2024; Mialonet al.,2023)similarly do not vary clarification timing.

2.2Theoretical Foundations Predict Timing Should Matter

The decision to seek clarification mid-trajectory is a special case of the value-of-information (VOI) problem(Lindley,1956; Howard,1966): should the agent pay a cost (interruption, latency) to reduce uncertainty before acting? Russell and Wefald(Russell and Wefald,1991)formalize this as metareasoning, and the active-learning paradigm(Settles,2009)studies when to query an oracle to maximize learning efficiency. In HCI, Adamczyk and Bailey(Adamczyk and Bailey,2004)demonstrate that task outcomes depend onwhenan interruption occurs, and Mark et al.(Market al.,2008)quantify the recovery cost of poorly timed interruptions. Recent work operationalizes these ideas for LLM agents: Rao and Daumé(Rao and Daumé III,2018)apply expected value of perfect information to rank clarification questions, Dong et al.(Donget al.,2026)propose a decision-theoretic framework, Kobalczyk et al.(Kobalczyket al.,2025)cast clarification as Bayesian experimental design, Suri et al.(Suriet al.,2025)use structured uncertainty for tool-calling agents, Lumer et al.(Lumeret al.,2025a)optimize short-term memory management across multi-turn tool interactions, and PRISM(Fuet al.,2026)learns a cost-sensitive gate for proactive interventions. These frameworks all predict that clarification value should depend on timing, yet none provides the empirical measurements our forced-injection experiments produce.

3Hypotheses: Trajectory Commitment

We develop a simple model predictingwhyclarification timing should interact with information dimension. A task is specified by a parameter vectorθ=(θgoal,θinput,θcon,θctx)\theta=(\theta_{\text{goal}},\theta_{\text{input}},\theta_{\text{con}},\theta_{\text{ctx}}), one component per dimension. The agent observes an underspecified versionθ~\tilde{\theta}in which one componentθd\theta_{d}is missing, and executes a trajectoryτ=(a1,…,aT)\tau=(a_{1},\ldots,a_{T}). At injection timett, the agent receives the true value ofθd\theta_{d}. We define thecommitmentCd​(t)∈[0,1]C_{d}(t)\in[0,1]as the fraction of actionsa1,…,ata_{1},\ldots,a_{t}that causally depend on dimensiondd. The value of clarification at timettis bounded by the recoverable portion:

VOId​(t)≤VOId​(0)⋅(1−Cd​(t)),\text{VOI}_{d}(t)\;\leq\;\text{VOI}_{d}(0)\cdot\bigl(1-C_{d}(t)\bigr),(1)since committed actions cannot be undone. The shape ofCd​(t)C_{d}(t)thus upper-bounds the timing decay profile for each dimension; the actual VOI may decay faster if reconciliation costs exceed the recoverable portion. We argue thatCd​(t)C_{d}(t)grows at different rates due to the causal structure of typical workflows:Goalandcontextcondition all subsequent actions: early approach selection cascades through the entire trajectory, predicting concave, front-loaded commitment.Inputaffects only data-dependent steps; agents can partially infer inputs through exploration, predicting approximately linear commitment.Constraintsimpose rules that may be invisible until violated; actions taken without constraint knowledge may actively conflict with it, so late injection can bedisruptive, with reconciliation cost exceeding the information’s value.

Table 1:Dataset and experimental setup. Action budgets are oracle-calibrated per model and variant. Total runs include all conditions and trials across available models.Table 2:Frontier models evaluated. API identifiers are provided for reproducibility. Clarification archetypes are derived from the natural-ask protocol (Section4.4). A fourth model, DeepSeek V3.2, is evaluated on MCP-Atlas only (see Section4.2).These predictions yield two falsifiable hypotheses.H1(Dimension-dependent timing): Goal and context clarification are front-loaded (steep VOI decay), while input clarification decays gradually.H2(Constraint attenuation): On benchmarks where an oracle gap exists, constraint clarification yields VOI that declines with delay, with reconciliation costs offsetting some but not all of the information’s value. We test both in Section5.

4Experimental Design

4.1Overview and Hypotheses

We test the hypotheses derived from the trajectory commitment model (Section3):H1predicts that goal and context clarification are front-loaded while input clarification decays gradually, andH2predicts that constraint clarification yields VOI that declines with delay on benchmarks where an oracle gap exists, with reconciliation costs offsetting some but not all of the information’s value. We additionally test whether, for each dimension, there exists apoint of no returnbeyond which clarification no longer improves over the no-clarification baseline.

For each of 84 underspecified task variants (full breakdown in AppendixA.10), we run seven conditions with three trials per condition across available models (four on MCP-Atlas, three on TheAgentCompany and SWE-Bench Pro), yielding over 6,000 total experiment runs across all conditions, models, and trials (AppendixA.8). The seven conditions are: (1)oracle, where the agent receives the fully-specified original prompt (upper bound); (2)no-clarification, where the agent receives the underspecified prompt and must complete the task without additional information (lower bound); and (3–7) fiveinjectionconditions at 10%, 30%, 50%, 70%, and 90% of the trajectory, where the missing information is provided mid-execution.

Figure 2:Natural ask protocol. Three models each run 100 sessions on underspecified TheAgentCompany tasks with theask_usertool enabled. We report two distinct percentages: theask rate(fraction of sessions in which the model asked at least once) and themean ask timing(how far through the trajectory the first ask occurred, as a percentage of total actions). GPT-5.2 asks frequently (ask rate 52%, mean timing 43%), Claude Sonnet 4.5 asks selectively (ask rate 23%, mean timing 50%), and Gemini 3 Flash never asks (ask rate 0%).

4.2Benchmarks and Models

We draw our 84 variants from a stratified subset of LHAW’s(Puet al.,2026)underspecified tasks, spanning three benchmarks that cover distinct agent domains (Table1):

MCP-Atlas(Bandiet al.,2026; Lumeret al.,2025b,c)(36 variants, 4 models) evaluates tool-use competency across real MCP servers with short trajectories (6–20 actions). All four information dimensions are represented. This is the only benchmark evaluating all four models including DeepSeek V3.2, as its standard tool-calling APIs require no benchmark-specific harness.

TheAgentCompany(Xuet al.,2024)(30 variants, 3 models) tests enterprise workflows on a simulated corporate platform with longer trajectories (6–49 actions) involving web, database, and tool interactions. Three dimensions are covered (goal, constraint, input). DeepSeek V3.2 is excluded due to harness integration limitations.

SWE-Bench Pro(Scale AI,2025)(18 variants, 3 models) evaluates code-repair tasks on real GitHub issues (1–121 actions). DeepSeek V3.2 is excluded; the remaining three models each have both injection and baseline data, yieldingn=12n{=}12–31 (variant, model) units per injection cell depending on dimension (AppendixA.7).

LHAW defines four information dimensions:goal(unclear deliverable),constraint(missing rules or thresholds),input(unspecified data source), andcontext(absent domain knowledge). Variants are stratified by dimension and ambiguity class:outcome-critical(missing information causes failure),divergent(valid but unintended output), andbenign(agent succeeds despite the gap).

4.3Forced Injection Protocol

Because tasks range from 6 to 121 actions, we calibrate injection points per model and task. For modelmmand variantvv, letBm,vB_{m,v}denote the mean oracle trajectory length (rounded to the nearest integer). The injection action for timingt∈{0.1,0.3,0.5,0.7,0.9}t\in\{0.1,0.3,0.5,0.7,0.9\}isainject=max⁡(1,⌊Bm,v⋅t⌋)a_{\text{inject}}=\max(1,\lfloor B_{m,v}\cdot t\rfloor). For SWE-Bench Pro tasks whereBm,v=1B_{m,v}=1, all five injection conditions collapse to action 1; these are retained to preserve sample size but contribute zero timing variance.

At actionainjecta_{\text{inject}}, a synthetic user message containing the ground-truthremoved_segmentsis inserted at the next clean turn boundary (AppendixA.11,A.12). Theask_usertool is disabled, so clarification arrives only through the injection. Messages read naturally (e.g., “By the way, I should have mentioned: the target format is CSV, not JSON.”). Each action corresponds to one tool call or one agent message turn.

4.4Natural Ask Protocol

To assess whether models naturally ask within the optimal timing window, we run a complementary 300-session study on TheAgentCompany tasks with theask_userMCP tool enabled. Each of three models (Claude Sonnet 4.5, GPT-5.2, Gemini 3 Flash) completes 100 sessions across seven base tasks (a stratified subset of the 30 TAC variants, selected to cover goal-dimension and input-dimension variants in roughly equal proportion across all three ambiguity classes). When invoked, a simulated user responds with the ground-truth missing information matching the forced-injection content. We record the action step of eachask_usercall as a percentage of total trajectory length (Figure2).

Figure 3:MCP-Atlas VOI curves by information dimension (n=5n{=}5–28 per dimension; see subplot titles). Goal clarification is dramatically front-loaded (0.78 at 10% vs. 0.40 NC), input shows gradual decline, and constraint shows minimal benefit over baseline. Dashed: no-clarification baseline; dotted: oracle. Per-benchmark curves and aggregate curves appear in AppendixA.1andA.9.

4.5Metrics

We evaluate with four complementary metrics:

[email protected] the HumanEval formulation(Chenet al.,2021), we compute the probability that at least one ofk=3k=3trials succeeds for each (variant, model, condition) cell. We report the mean pass@3 across units within each experimental group.

Wasted compute.For injection conditions, we count pre-injection actions whose effects are absent from the oracle (fully-specified) trace, normalized by total pre-injection actions. This quantifies the fraction of early work rendered useless by late clarification.

Actions absent from the oracle trace may still be indirectly useful (e.g., exploratory schema queries that inform the agent’s understanding). We therefore treat this metric as a conservative upper bound on waste, appropriate for quantifying the cost ceiling of late clarification rather than claiming every non-oracle action is worthless. Qualitative analysis (AppendixA.6) confirms that some “wasted” exploration partially compensates for missing specifications, consistent with input’s gradual VOI decline.

Point of no return.The latest injection timing at which pass@3 significantly exceeds the no-clarification baseline (one-sided permutation test,p<0.05p<0.05), stratified by dimension and ambiguity class.

Cross-model consistency.Kendall’sτ\taurank correlation(Kendall,1938)of per-variant pass@3 vectors across model pairs, measuring whether timing effects are task-intrinsic or model-dependent.

Figure 4:Wasted compute across all benchmarks. Left: MCP-Atlas and TheAgentCompany (fraction of pre-injection actions absent from the oracle trace). Right: SWE-Bench Pro (absolute action count; different scale due to longer trajectories). Later injection consistently produces more wasted work.Table 3:Cross-model Kendall’sτ\taurank correlation. Left: TheAgentCompany only (3 models, filtered to complete-data pairs). Right: all benchmarks combined (4 models).(a)TheAgentCompany (b)All benchmarks combined

5Results

5.1VOI Curves Are Dimension-Dependent (H1)

Figure3presents VOI curves on MCP-Atlas, the benchmark with the strongest timing signal (n=5n{=}5–28 units per dimension; complete per-benchmark values in Appendix Table4). Clarification timing interacts strongly with information dimension, partially confirming H1 (the goal-front-loading prediction is strongly supported on MCP-Atlas; the context prediction is directionally consistent but underpowered atn=17n{=}17across two benchmarks):

Goal.Exhibits dramatic front-loading. Injection at 10% of the trajectory recovers near-oracle performance (pass@3 of 0.78 vs. oracle 0.80, NC 0.40). The benefit decays steeply: Inj-30 = 0.50 (above NC), Inj-50 = 0.44 (above NC), Inj-70 = 0.39 (≈\approxNC), Inj-90 = 0.39. By 70% of the trajectory, goal injection no longer improves over no-clarification. This confirms that goal information has a narrow early window of high value.

Input.Shows gradual decline (0.46 at Inj-10, 0.36 at Inj-30 and Inj-50, 0.32 at Inj-70, 0.25 at Inj-90; NC = 0.33, oracle = 0.57). The early window (Inj-10 through Inj-50) recovers most of the oracle gap, but by Inj-70 input clarification falls to the NC baseline and by Inj-90 sits well below it. The flatter slope relative to goal confirms that input information is recoverable for longer than goal information, consistent with agents partially compensating through environment exploration, but this recovery window extends only through approximately 50% of the trajectory.

Figure 5:Natural ask timing overlaid on aggregate VOI curves for goal and input dimensions. Vertical lines indicate mean first-ask timing: GPT-5.2 (purple, 43%) asks within the early-injection window for input but late for goal; Claude (orange, 50%) asks at midpoint. Gemini never asks. The shaded region marks the optimal window for goal clarification (defined as the range of injection timings whose pass@3 is within 5 percentage points of the Inj-10 maximum).Constraint.Constraint behavior depends on whether an oracle gap exists. On MCP-Atlas, Oracle and NC are identical (both 0.12), so the benchmark is uninformative for H2 (there is no information value to attenuate), and a flat injection curve (0.13–0.17 across timings) is consistent with both H2 and the null hypothesis. On SWE-Bench Pro, where the oracle gap is substantial (0.81 vs. 0.56 NC), constraint injection shows genuine declining benefit (0.81 at Inj-10, declining to 0.68 at Inj-90; Appendix Table4), supporting a weaker form of H2: reconciliation costs reduce constraint VOI relative to the oracle, but injected constraint information remains above the no-clarification baseline at all tested timings (0.68>>0.56). The strong form of H2 (below-baseline disruption) is not supported by either benchmark. Constraint-type clarification should ideally be provided before execution begins rather than mid-trajectory.

Context.Evidence comes from two benchmarks with limited variant diversity: MCP-Atlas (n=5n{=}5) shows strong early-injection benefit (0.80 at 10% vs. 0.60 NC), and SWE-Bench Pro (n=12n{=}12) is essentially flat across timings (0.92 at most injection points, with Oracle = 0.67 and NC = 0.75; the NC>>Oracle inversion reflects sampling variability; see AppendixA.7). The combined sample (n=17n{=}17) is too small for reliable conclusions about timing structure for context, though the MCP-Atlas signal is consistent with H1’s front-loading prediction.

Cross-benchmark patterns.On TheAgentCompany, floor effects (oracle≤\leq29%) attenuate timing signals. Goal injection conditions all sit within 4 pp of NC (0.12), consistent with sampling variability atn=65n{=}65rather than injection-induced disruption. Input shows mild benefit at intermediate timings (0.21–0.25 vs. 0.19 NC). Constraint is at floor (0% across all conditions). SWE-Bench Pro results (AppendixA.1; caveats inA.7) are directionally consistent. Aggregate curves appear in AppendixA.9.

Point of no return.We find that clarification benefit is continuous rather than threshold-based: only outcome-critical variants show a statistically significant recovery point, at 30% of the trajectory (p<0.05p<0.05; AppendixA.5). Earlier is always better, but there is no sharp cutoff after which clarification becomes useless.

5.2Wasted Compute

Wasted compute (fraction of pre-injection actions absent from the oracle trace, an upper bound on discarded work) increases steadily with injection delay across all three benchmarks (Figure4; AppendixA.3). On TheAgentCompany, waste rises from 0.0% at Inj-10 to 21.7% at Inj-90. On MCP-Atlas, it ranges from 38.4% to 52.9% (higher baseline reflecting shorter trajectories). SWE-Bench Pro confirms the pattern: 0.7 to 10.4 wasted actions.

5.3Cross-Model Consistency

Table3reports Kendall’sτ\taurank correlations between model pairs. On TheAgentCompany (3 models, balanced variant set), correlations range from 0.78 to 0.87, indicating strong agreement on which variants benefit from early clarification. On the combined dataset across all benchmarks (4 frontier models), Claude-Gemini achievesτ=0.67\tau=0.67and all model pairs exceed 0.34 (p<0.01p<0.01for all pairs; see AppendixA.4). These results support the interpretation that timing effects are predominantly task-intrinsic: the same variants benefit from early injection regardless of which model executes them.

5.4Natural Ask Overlay

Our natural-ask analysis of 300 TheAgentCompany sessions reveals stark differences in model clarification behavior (AppendixA.2provides per-model statistics and timing distributions). GPT-5.2 asks in 52% of sessions with a mean first-ask timing of 43% through the trajectory. Claude Sonnet 4.5 asks in 23% of sessions at a mean timing of 50%. Gemini 3 Flash never asks (0% ask rate across all 100 sessions). These archetypes are consistent with LHAW’s independent finding, under a different protocol, that GPT-5.2 over-clarifies with the lowest per-question efficiency while Gemini models under-clarify(Puet al.,2026).

Overlaying these natural timings on the forced-injection VOI curves reveals timing alignment gaps (Figure5). GPT-5.2’s asking at 43% is past the goal optimum (10%) but within the input window; Claude’s 50% is suboptimal for goal but reasonable for input; Gemini forgoes all benefit. Notably, timing alignment alone does not predict success: Claude achieves higher per-session success (11%) than GPT-5.2 (3%) despite asking later and less often (AppendixA.2), suggesting that question quality may matter more than frequency, though this between-model comparison cannot isolate question quality from other model-level differences, and we treat it as a hypothesis rather than a controlled finding.

6Limitations

This study establishes thedemand sideof clarification timing (how much does task performance benefit from receiving information at each trajectory point?) but does not address thesupply side: making agents recognize ambiguity and ask at the right moment. Building supply-side mechanisms on top of these VOI curves is an important next step. The strongest timing signals come from MCP-Atlas (n=5n{=}5–28 per dimension); TheAgentCompany provides weaker signal due to floor effects, and SWE-Bench Pro has moderate per-cell sizes (n=12n{=}12–31). The natural-ask protocol covers only TheAgentCompany (300 sessions). A behavioral confound exists between protocols: forced injection disablesask_userwhile natural-ask enables it, so agents may plan differently when they know they cannot ask; the forced-injection VOI curves should be interpreted as upper bounds on information value.

7Conclusion

We present the first empirical VOI curves for clarification timing in long-horizon agent workflows. Timing effects are strongly dimension-dependent: goal clarification is front-loaded (≤{\leq}10% window), input degrades gradually (recoverable through∼{\sim}50%), and constraint benefit depends on the oracle gap, with cross-model consistency confirming these profiles are task-intrinsic. No current frontier model asks within the optimal window. These findings yield two actionable guidelines: (1) agents should validate goals within the first few actions; (2) input queries can be deferred slightly longer but should be raised within the first half of the trajectory. More broadly, the forced-injection methodology generalizes to any agent benchmark with underspecified tasks and provides the empirical demand curves that existing theoretical frameworks(Howard,1966; Donget al.,2026; Kobalczyket al.,2025)require but have lacked. Building timing-aware clarification policies(Senet al.,2026)on top of these curves is a natural next step.

Acknowledgments and Disclosure of Funding

We thank the LHAW, MCP-Atlas, TheAgentCompany, and SWE-Bench Pro teams for their open-source benchmarks. We also thank our colleagues at PricewaterhouseCoopers for valuable feedback on earlier drafts.

References

• P. D. Adamczyk and B. P. Bailey (2004)If not now, when? the effects of interruption at different moments within task execution.InProceedings of the SIGCHI Conference on Human Factors in Computing Systems,pp. 271–278.Cited by:§1,§2.2.
• M. Aliannejadi, J. Kiseleva, A. Chuklin, J. Dalton, and M. Burtsev (2020)ConvAI3: generating clarifying questions for open-domain dialogue systems (ClariQ).arXiv preprint arXiv:2009.11352.Cited by:§2.1.
• C. Bandi, B. Hertzberg, G. Boo, T. Polakam, J. Da, S. Hassaan, M. Sharma, A. Park, E. Hernandez, D. Rambado, I. Salazar, R. Cruz, C. Rane, B. Levin, B. Kenstler, and B. Liu (2026)MCP-Atlas: a large-scale benchmark for tool-use competency with real MCP servers.arXiv preprint arXiv:2602.00933.Cited by:§1,§4.2.
• M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman,et al.(2021)Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374.Cited by:§4.5.
• J. A. Clouse (1996)On integrating apprentice learning and reinforcement learning.Ph.D. Thesis,University of Massachusetts Amherst.Cited by:§2.1.
• Y. R. Dong, T. Hu, Z. Hui, C. Zhang, I. Vulić, A. Bobu, and N. Collier (2026)Value of information: a framework for human-agent communication.arXiv preprint arXiv:2601.06407.Cited by:§2.2,§7.
• M. Elfeki, T. Trinh, G. Luo, K. Luu, N. Hunt, E. Hernández, N. Marwaha, Y. Y. He, C. Wang, F. Carabedo, A. Castillo, and B. Liu (2026)HiL-Bench: do agents know when to ask for help?.arXiv preprint arXiv:2604.09408.Cited by:§1,§2.1.
• Y. Fu, X. Tan, T. Hao, C. Zhan, and X. Qiu (2026)PRISM: festina lente proactivity – risk-sensitive, uncertainty-aware deliberation for proactive agents.InInternational Conference on Learning Representations,Cited by:§2.2.
• R. A. Howard (1966)Information value theory.IEEE Transactions on Systems Science and Cybernetics2(1),pp. 22–26.External Links:DocumentCited by:§1,§2.2,§7.
• Z. Hu, C. Liu, X. Feng, Y. Zhao, S. Ng, A. T. Luu, J. He, P. W. Koh, and B. Hooi (2024)Uncertainty of thoughts: uncertainty-aware planning enhances information seeking in large language models.InAdvances in Neural Information Processing Systems,Cited by:§2.1.
• C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan (2024)SWE-bench: can language models resolve real-world GitHub issues?.InInternational Conference on Learning Representations,Cited by:§1.
• S. Kadavath, T. Conerly, A. Askell, T. Henighan, D. Drain, E. Perez, N. Schiefer, Z. Hatfield-Dodds, N. DasSarma, E. Tran-Johnson,et al.(2022)Language models (mostly) know what they know.arXiv preprint arXiv:2207.05221.Cited by:§2.1.
• M. G. Kendall (1938)A new measure of rank correlation.Biometrika30(1/2),pp. 81–93.Cited by:§4.5.
• M. Kobalczyk, N. Astorga, T. Liu, and M. van der Schaar (2025)Active task disambiguation with LLMs.InInternational Conference on Learning Representations,Cited by:§2.2,§7.
• L. Kuhn, Y. Gal, and S. Farquhar (2023)CLAM: selective clarification for ambiguous questions with generative language models.InICML 2023 Workshop on Structured Probabilistic Inference and Generative Modeling,Cited by:§2.1.
• B. Z. Li, B. Kim, and Z. Wang (2025)QuestBench: can models ask the right questions?.arXiv preprint arXiv:2503.22674.Cited by:§2.1.
• D. V. Lindley (1956)On a measure of the information provided by an experiment.Annals of Mathematical Statistics27(4),pp. 986–1005.External Links:DocumentCited by:§2.2.
• G. Liuet al.(2025)CaRT: teaching LLM agents to know when they know enough.arXiv preprint arXiv:2510.08517.Cited by:§2.1.
• X. Liu, H. Yu, H. Zhang, Y. Xu, X. Lei, H. Lai, Y. Gu, H. Ding, K. Men, K. Yang,et al.(2024)AgentBench: evaluating LLMs as agents.InInternational Conference on Learning Representations,Cited by:§2.1.
• E. Lumer, A. Gulati, F. Nizar, D. Hedroits, A. Mehta, H. Hwangbo, V. K. Subbiah, P. H. Basavaraju, and J. A. Burke (2026a)Tool and agent selection for large language model agents in production: a survey.Preprints.External Links:Document,LinkCited by:§1.
• E. Lumer, A. Gulati, V. K. Subbiah, P. H. Basavaraju, and J. A. Burke (2025a)Memtool: optimizing short-term memory management for dynamic tool calling in llm agent multi-turn conversations.arXiv preprint arXiv:2507.21428.Cited by:§2.2.
• E. Lumer, A. Gulati, V. K. Subbiah, P. H. Basavaraju, and J. A. Burke (2025b)Scalemcp: dynamic and auto-synchronizing model context protocol tools for llm agents.InInternational Joint Conference on Computational Intelligence,pp. 23–42.Cited by:§4.2.
• E. Lumer, F. Nizar, A. Gulati, P. H. Basavaraju, and V. K. Subbiah (2025c)Tool-to-agent retrieval: bridging tools and agents for scalable llm multi-agent systems.arXiv preprint arXiv:2511.01854.Cited by:§4.2.
• E. Lumer, F. Nizar, A. Jangiti, K. Frank, A. Gulati, M. Phadate, and V. K. Subbiah (2026b)Don’t break the cache: an evaluation of prompt caching for long-horizon agentic tasks.arXiv preprint arXiv:2601.06007.Cited by:§1.
• G. Mark, D. Gudith, and U. Klocke (2008)The cost of interrupted work: more speed and stress.InProceedings of the SIGCHI Conference on Human Factors in Computing Systems,pp. 107–110.Cited by:§1,§2.2.
• G. Mialon, C. Fourrier, C. Swift, T. Wolf, Y. LeCun, and T. Scialom (2023)GAIA: a benchmark for general AI assistants.arXiv preprint arXiv:2311.12983.Cited by:§2.1.
• S. Min, V. Zhong, L. Zettlemoyer, and H. Hajishirzi (2020)AmbigQA: answering ambiguous open-domain questions.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP),pp. 5783–5797.Cited by:§2.1.
• S. Y. Min, Y. Wu, J. Sun, M. Kaufmann, F. Tajwar, Y. Bisk, and R. Salakhutdinov (2025)Self-regulation and requesting interventions.arXiv preprint arXiv:2502.04576.Cited by:§2.1.
• J. Panet al.(2025)When benchmarks talk: re-evaluating code generation with interactive tasks.arXiv preprint arXiv:2502.18413.Cited by:§2.1.
• B. Plaut, H. Zhu, and S. Russell (2025)Avoiding catastrophe in online learning by asking for help.InInternational Conference on Machine Learning,Cited by:§2.1.
• G. Pu, M. S. Lee, U. M. Sehwag, D. J. Lee, B. Zhu, Y. Maurya, M. Raghavendra, Y. Xue, and S. M. Denton (2026)LHAW: controllable underspecification for long-horizon tasks.arXiv preprint arXiv:2602.10525.Cited by:§A.10,§A.11.4,§1,§2.1,§4.2,§5.4.
• C. Qianet al.(2025)UserBench: an interactive gym environment for user-centric agents.arXiv preprint arXiv:2507.22034.Cited by:§2.1.
• S. Rao and H. Daumé III (2018)Learning to ask good questions: ranking clarification questions using neural expected value of perfect information.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),pp. 2737–2746.Cited by:§2.2.
• S. Ross, G. J. Gordon, and J. A. Bagnell (2011)A reduction of imitation learning and structured prediction to no-regret online learning.InProceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS),pp. 627–635.Cited by:§2.1.
• S. J. Russell and E. Wefald (1991)Do the right thing: studies in limited rationality.MIT Press,Cambridge, MA.Cited by:§2.2.
• Scale AI (2025)SWE-Bench Pro: a more challenging benchmark for real-world software engineering.arXiv preprint arXiv:2509.16941.Cited by:§4.2.
• S. Sen, E. Lumer, A. Gulati, and V. K. Subbiah (2026)Chronos: temporal-aware conversational agents with structured event retrieval for long-term memory.arXiv preprint arXiv:2603.16862.Cited by:§7.
• B. Settles (2009)Active learning literature survey.Technical reportTechnical ReportTR-1648,University of Wisconsin-Madison, Department of Computer Sciences.Cited by:§2.2.
• M. Suri, P. Mathur, N. Lipka, F. Dernoncourt, R. A. Rossi, and D. Manocha (2025)Structured uncertainty guided clarification for LLM agents.arXiv preprint arXiv:2511.08798.Cited by:§2.2.
• A. Tamkin, K. Handa, A. Shrestha, and N. Goodman (2023)Task ambiguity in humans and language models.InInternational Conference on Learning Representations,Cited by:§2.1.
• H. Trivedi, T. Khot, M. Hartmann, R. Manber, V. Dong, E. Li, S. Harber, H. Fang, A. Mishra, D. Radev, and A. Sabharwal (2024)AppWorld: a controllable world of apps and people for benchmarking interactive coding agents.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,Note:Best Resource Paper AwardCited by:§2.1.
• S. Wu, M. Galley, B. Peng, H. Cheng, G. Li, Y. Dou, W. Cai, J. Zou, J. Leskovec, and J. Gao (2025)CollabLLM: from passive responders to active collaborators.InInternational Conference on Machine Learning,Note:Outstanding Paper AwardCited by:§2.1.
• T. Xie, D. Zhang, J. Chen, X. Li, S. Zhao, R. Cao, T. J. Hua, Z. Cheng, D. Shi, Z. Lu,et al.(2024)OSWorld: benchmarking multimodal agents for open-ended tasks in real computer environments.InAdvances in Neural Information Processing Systems,Cited by:§2.1.
• F. F. Xu, Y. Shi, Y. Lin, X. Yang, T. Xie, B. Wang, H. Peng, S. Yao, Q. Shi, S. Wang,et al.(2024)TheAgentCompany: benchmarking LLM agents on consequential real world tasks.arXiv preprint arXiv:2412.14161.Cited by:§1,§4.2.
• S. Yaoet al.(2024)τ\tau-Bench: a benchmark for tool-agent-user interaction in real-world domains.arXiv preprint arXiv:2406.12045.Cited by:§2.1.
• S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao (2023)ReAct: synergizing reasoning and acting in language models.InInternational Conference on Learning Representations,Cited by:§1.
• X. Zhang, Y. Deng, Z. Ren, S. Ng, and T. Chua (2024)Ask-before-plan: proactive language agents for real-world planning.InFindings of the Association for Computational Linguistics: EMNLP 2024,Cited by:§2.1.
• S. Zhou, F. F. Xu, H. Zhu, X. Zhou, R. Lo, A. Sridhar, X. Cheng, T. Ou, Y. Bisk, D. Fried, U. Alon, and G. Neubig (2024)WebArena: a realistic web environment for building autonomous agents.InInternational Conference on Learning Representations,Cited by:§2.1.

Appendix AExtended Results

A.1Per-Benchmark VOI Curves

Table4presents the complete pass@3 values for all (benchmark, dimension, condition) cells. Figures6–8visualize these as per-benchmark VOI curves.

Table 4:Complete pass@3 results by benchmark, dimension, and condition.nndenotes injection-condition (variant, model) units per dimension; only pairs for which all five injection conditions and the oracle baseline completed successfully are included (pairs with failures in any condition are excluded). For multi-dimension variants (Table10), each pair is attributed to its primary dimension only. Oracle and NC cells may include up to 4 additional baseline-only units (per-cellnnfor Oracle/NC is thereforen+kn+kwherek∈[0,4]k\in[0,4]; exact counts are available in the released code). TheAgentCompany uses the 3-model subset (see Section4.2).Figure 6:MCP-Atlas VOI curves by dimension (expanded view with per-model error indicators; cf. the pooled summary in main text Figure3). Goal shows dramatic front-loading (0.78 at 10% vs. 0.40 NC), input shows gradual decline, constraint is flat near zero, and context (limitedn=5n{=}5) shows strong early benefit. Dashed: no-clarification baseline; dotted: oracle.Figure 7:TheAgentCompany VOI curves by dimension. Success rates are substantially lower than MCP-Atlas (oracle≤\leq29% for non-constraint dimensions). Goal shows no reliable timing effect (all conditions within 4 percentage points of baseline). Input shows a mild benefit at intermediate timings (21–25% at Inj-30/50/70 vs. 19% NC). Constraint is at floor (0% across all conditions).Figure 8:SWE-Bench Pro VOI curves by dimension, pooled across three models (n=12n{=}12–31 per injection cell). All four dimensions show injection benefit over no-clarification, with input and goal exhibiting the clearest declining VOI pattern. Context has limited variant diversity (n=12n{=}12).Benchmark-specific interpretation.The three benchmarks represent a spectrum of task characteristics that explain their distinct VOI patterns:

• •MCP-Atlastasks are short (6–20 actions), tool-focused, and typically have a single correct approach. This creates high commitment rates: choosing the wrong tool chain in the first few actions wastes most of the trajectory. Goal clarification is therefore extremely front-loaded.
• •TheAgentCompanytasks are longer (6–49 actions) and involve multiple interacting systems (web, database, file system). The lower baselines reflect genuine task difficulty; even oracle-informed agents achieve only 20–29% pass@3 on non-constraint dimensions. The weak timing signal for goal suggests that these tasks have inherently high stochasticity: success depends on factors beyond information completeness (e.g., correct navigation of web interfaces).
• •SWE-Bench Protasks vary enormously in difficulty (1–121 actions). Input shows a gradual decline from 1.00 (Inj-10) to 0.75 (Inj-90), consistent with code-generation patterns: once an agent commits to an implementation approach (choosing file locations, function signatures, test strategies), switching costs increase steadily.

A.2Natural Ask Timing Details

The 300 natural-ask sessions span seven TheAgentCompany base tasks. Table5summarizes per-model ask behavior.

Table 5:Natural ask behavior by model (100 sessions each). Ask rate is the fraction of sessions with≥\geq1 ask call. Mean timing is the trajectory-relative position of the first ask.GPT-5.2 averages 1.71 ask calls per asking session (max 5), indicating a pattern of iterative clarification. Claude always asks exactly once when it asks at all, suggesting a more deliberate single-query strategy. The distribution of first-ask timings is concentrated: 72% of all first-ask events occur between 25% and 50% of the trajectory.

Figure 9:Distribution of first-ask timing by model. GPT-5.2 asks frequently (52% of sessions) with timing concentrated between 25–50% of the trajectory. Claude asks selectively (23%) with a tight cluster around 50%. Gemini never asks. Dashed lines indicate mean first-ask timing.Figure 10:Per-task ask rate across all models. Task complexity appears to influence asking propensity independently of model architecture, with data-analysis tasks eliciting higher ask rates than simpler administrative tasks.Qualitative analysis of ask content.GPT-5.2’s questions tend to be specific and actionable (e.g., “Should I use the FY25 Q1 data or the calendar year Q1?”), while Claude’s questions are broader (e.g., “Could you clarify what format you’d like the output in?”). Despite asking far more often, GPT-5.2 achieves substantially lower per-session success (3%) than Claude (11%), indicating that asking frequency does not predict task success. Claude’s selective single-question strategy outperforms GPT-5.2’s iterative approach, suggesting that question quality and selectivity may matter more than volume, though we cannot isolate this from other model-level differences. This is consistent with the VOI framework: later and repeated questions yield diminishing returns, and poorly targeted questions may provide no actionable benefit even when asked at an appropriate trajectory point.

A.3Wasted Compute by Benchmark

Table6reports mean wasted compute fractions by benchmark and injection timing. On TheAgentCompany, wasted compute follows a clean linear pattern reflecting the longer trajectories where waste accumulates gradually. On MCP-Atlas, the higher baseline (38% even at injection-10) reflects shorter trajectories where even a single early action pursuing the wrong path counts as waste relative to the oracle trace.

Table 6:Wasted compute by benchmark and injection timing. MCP-Atlas and TheAgentCompany report the fraction of pre-injection actions absent from the oracle trace; SWE-Bench Pro reports absolute action counts due to longer, variable-length trajectories.The MCP-Atlas pattern of high initial waste (38% at injection-10) that plateaus at 53% reflects a structural property of short tool-calling trajectories: even early injection requires the agent to restart its approach, and the first 1–2 actions are often already committed. In contrast, TheAgentCompany’s linear growth from 0% reflects that the first few actions in enterprise workflows (opening a browser, navigating to a page) are typically approach-independent and not wasted regardless of the eventual clarification.

A.4Cross-Model Correlation Details

Figure 11:Kendallτ\taucorrelation heatmaps. Left: TheAgentCompany only (3 models, balanced variant set). Right: all benchmarks combined (4 frontier models). Allpp-values<0.01<0.01except where noted.Table7reports thepp-values for all model-pair correlations. For the combined dataset, all 4-model pairs achieve significance atp<0.01p<0.01. For the TAC-only analysis, all pairs are significant atp<10−5p<10^{-5}, reflecting the larger number of comparable variant-timing combinations (each model runs all 30 TAC variants across 5 injection conditions).

Table 7:Kendallτ\taupp-values for all model pairs. Combined: all benchmarks (4 models). TAC: TheAgentCompany only (3 models, filtered to complete-data pairs).(a)TACpp-values (b)Combinedpp-values

The higher TAC correlations (0.78–0.87) compared to the combined dataset (0.34–0.67) likely reflect two factors: (1) the TAC subset uses exactly the same variant set and injection protocol across all three models (ensuring apple-to-apple comparison), and (2) the combined dataset includes MCP-Atlas with 4 models (introducing DeepSeek which is only available on that benchmark) and SWE-Bench Pro where high overall pass rates compress the VOI range, attenuating rank correlations.

A.5Point of No Return Analysis

Figure 12:Latest useful injection point by dimension and ambiguity class. Only outcome-critical variants for goal and constraint dimensions show a statistically significant recovery point (at 30% of trajectory). Blank bars indicate no significant point of no return was detected.Table8reports the point-of-no-return analysis. We define the “latest useful injection point” as the latest timing at which pass@3 significantly exceeds the no-clarification baseline (one-sided permutation test with 10,000 permutations,p<0.05p<0.05, Bonferroni-corrected for 5 timing comparisons per cell).

Table 8:Point of no return: latest injection timing with significant improvement over no-clarification (p<0.05p<0.05, Bonferroni-corrected). “–” indicates no timing achieves significance.The absence of significant points for input and context dimensions is consistent with the VOI curves: input shows gradual decline (never sharply crossing the baseline), and context has too few samples (n=5n{=}5) for statistical power. For goal and constraint, the 30% cutoff aligns with the commitment model’s prediction that these dimensions have steep early commitment rates.

A.6Qualitative Trajectory Analysis

To illustrate how timing interacts with agent behavior, we present representative trajectory patterns for each dimension:

Goal (MCP-Atlas, early injection helps).A task requires exporting data “in a format suitable for spreadsheet analysis.” Without clarification, all three models default to JSON output. With goal clarification at 10% (“the user wants CSV format”), the agent correctly outputs CSV from the start. At 50%, two of three models have already committed to JSON parsing pipelines and must discard 3–5 actions to restart. At 90%, all models have completed the task in the wrong format.

Input (MCP-Atlas, gradual decline).A task requires querying “the customer database” but doesn’t specify which table contains customer records. Without clarification, agents explore the schema (listing tables, examining columns), eventually finding the correct table through trial and error. This exploration is “wasted” relative to the oracle trace but is informationally productive, as it narrows the search space. Early injection eliminates this exploration; late injection still saves the agent from querying the wrong table at the final step.

Constraint (TAC, disruptive injection).A task asks for a budget report but omits that “all figures must be in thousands.” Without this constraint, agents produce correct reports in raw units. When the constraint is injected at 50%, agents attempt to retroactively scale existing outputs, sometimes introducing rounding errors or formatting inconsistencies that reduce checkpoint scores below the unconstrained baseline.

A.7SWE-Bench Pro Limitations

SWE-Bench Pro results warrant additional caveats beyond those noted in Section6:

1. 1.Moderate sample sizes.Pooling across three models yieldsn=12n{=}12–31 (variant, model) units per injection cell depending on dimension. Per-model estimates remain noisy, but the pooled values in Table4are more stable than single-model estimates.
2. 2.Model coverage.Three models (Gemini 3 Flash, GPT-5.2, Claude Sonnet 4.5) have both injection and baseline data, enabling within-model comparisons. DeepSeek V3.2 is excluded from SWE-Bench Pro.
3. 3.Context dimension.Only 4 context-type variants are available in LHAW’s SWE-Bench Pro subset, yieldingn=12n{=}12units when pooled across models. The resulting estimates should be interpreted with caution given the limited variant diversity.

Additionally, multiple SWE-Bench Pro cells exhibit values above the Oracle baseline: Goal Inj-10 (0.86 vs. Oracle 0.83), Input Inj-10 (1.00 vs. 0.83), Input Inj-30 (0.92 vs. 0.83), Context Inj-10 (0.92 vs. 0.67), and Context NC (0.75 vs. 0.67). At sample sizes ofn=12n{=}12–31 per cell, these inversions are consistent with sampling variability. An additional contributing factor is that the injection messages provide explicit, structured guidance (e.g., “the target format is CSV”) that may be more salient than the same information embedded in the original task prompt; this is a known artifact of forced-injection protocols. None of these differences is statistically significant. We retain Oracle as the conceptual upper bound while noting that small-sample noise can produce localized inversions.

Despite these limitations, SWE-Bench Pro contributes to the aggregate picture by confirming that code-generation tasks exhibit steep commitment rates (consistent with the goal-dimension pattern on MCP-Atlas) and that oracle information substantially improves performance (0.81–0.83 oracle vs. 0.46–0.63 no-clarification for goal, constraint, and input dimensions across three models; see Table4).

A.8Total Runs and Computational Cost

Table9summarizes the total experimental scope.

Table 9:Total experimental trials by benchmark and protocol. Each forced-injection trial corresponds to one (variant, model, condition, trial-index) execution. Natural-ask sessions are single-trial.

A.9Aggregate VOI Interpretation

Figure 13:Aggregate VOI curves by information dimension (all benchmarks combined). Goal clarification is front-loaded, input shows gradual decline, and constraint injection provides only marginal benefit over no-clarification. Dashed lines indicate oracle (upper) and no-clarification (lower) baselines. See main text Figure3for the per-benchmark MCP-Atlas curves that drive the strongest signals.The aggregate curves in Figure13pool across benchmarks with substantially different baseline performance levels, weighting each (variant, model) unit equally. Because TAC contributes the majority of goal units (63%) with near-floor baselines, the aggregate attenuates the stronger timing signals visible in the per-benchmark curves (Figures6–8). The per-benchmark breakdowns should be consulted alongside this summary view.

A.10Dataset Details

Table10provides the full cross-tabulation of the 84 task variants used in this study, broken down by benchmark, information dimension(s), and ambiguity class. The subset is drawn from a larger pool of 101 variants in LHAW[Puet al.,2026]; 17 SWE-Bench Pro variants are excluded to manage computational costs. Selection was stratified to preserve dimension balance: excluded variants do not differ systematically from retained ones on dimension distribution or ambiguity class (see Section6).

Table 10:Task variant breakdown by benchmark, dimension, and ambiguity class. Multi-dimension variants (e.g., “goal+input”) have multiple segments removed simultaneously. Totals: 36 MCP-Atlas, 30 TheAgentCompany, 18 SWE-Bench Pro.

A.11Injection Protocol Details

A.11.1Injection Message Construction

Injection messages are constructed from a template system that maps each (dimension, subdimension) pair to a natural-language message pattern. Table11shows representative templates. For variants with multiple removed segments, the first segment uses the full template and subsequent segments are appended with connective phrases (“And {value}.” or “Also, {value}.”).

Table 11:Representative injection message templates by dimension and subdimension. The{value}placeholder is filled with the ground-truth content from the LHAW variant specification.Example injection messages from actual experiments:

• •Goal (format):“By the way, please give me the result in CSV format.”
• •Constraint (temporal + selection):“By the way, I should have mentioned: I’m looking at last month of 2022. Also, 0.50.”
• •Input (source):“By the way, you can find the data in the shared_drive/Q3_budget folder.”

A.11.2ask_user Tool Definition

For natural-ask experiments, theask_usertool is provided to models with the following schema (OpenAI function-calling format):

{
  "name": "ask_user",
  "description": "Ask the user a clarifying question to get
    more information about the task. Use this when the task
    is ambiguous or you need specific details to proceed.",
  "parameters": {
    "properties": {
      "question": {"type": "string",
        "description": "The clarifying question to ask."},
      "context": {"type": "string",
        "description": "Optional additional context.",
        "default": ""}
    },
    "required": ["question"]
  }
}

Additionally, the task prompt is extended with:

IMPORTANT: Your output will be checked by an auto-grader looking for exact answers. This task may be missing critical information. Use the ask_user tool to ask the user for any missing details.

When the model invokesask_user, a simulated user responds with the ground-truthremoved_segmentscontent, matching the injection message that would be used in the forced-injection condition.

A.11.3Oracle Budget Calibration

Injection timing is defined relative to an oracle-calibrated action budgetBm,vB_{m,v}for each (modelmm, variantvv) pair. The budget is computed as the mean trajectory length across 3 oracle-condition trials, rounded to the nearest integer:

Bm,v=round​(13​∑t=13|τm,v,toracle|)B_{m,v}=\text{round}\!\left(\frac{1}{3}\sum_{t=1}^{3}|\tau^{\text{oracle}}_{m,v,t}|\right)The injection action for timing fractionf∈{0.1,0.3,0.5,0.7,0.9}f\in\{0.1,0.3,0.5,0.7,0.9\}is:

ainject=max⁡(1,⌊Bm,v⋅f⌋)a_{\text{inject}}=\max\!\bigl(1,\;\lfloor B_{m,v}\cdot f\rfloor\bigr) Budget ranges reflect task diversity: MCP-Atlas tasks are short (6–20 actions), TheAgentCompany tasks are moderate (6–49 actions), and SWE-Bench Pro tasks can require extended code navigation and editing (1–121 actions).

A.11.4Reproducibility Parameters

All models are queried with temperature=0.0=0.0(greedy decoding) and a maximum output length of 4,096 tokens per turn. The three trials per (variant, model, condition) cell use fixed random seeds (0, 1, 2) for any stochastic API parameters. Total API cost for the 6,348 experimental runs was approximately $4,200 USD across all providers. Task success is determined by the LHAW automated grader[Puet al.,2026], which compares agent outputs against ground-truth answers using exact-match or rubric-based criteria depending on the benchmark (MCP-Atlas: tool-output match; TheAgentCompany: checkpoint scoring; SWE-Bench Pro: test-suite pass).

A.12Example Trajectory

Table12illustrates the structure of a single forced-injection trial. This MCP-Atlas example (variant688ba1...::V_S1_delete, GPT-5.2, injection at 10%) shows a 7-action trajectory with the injection arriving as a user message after step 1.

Table 12:Abridged trial trace (MCP-Atlas, injection at 10%, GPT-5.2). The injection message arrives as a synthetic user turn between steps 1 and 2. All subsequent actions incorporate the clarified information.Each trial JSON records:variant_id,model,condition,injection_point,actions(with per-stepaction_name,parameters,result,is_pre_injection),conversation(full chat history including the injection turn),task_success,total_actions,pre_injection_actions,post_injection_actions, andduration_seconds. All trial data will be released with the code.