Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories

Hugging Face Daily Papers Papers
error-localization deep-research-agents reliability benchmark claim-centric-auditing agent-trajectories

Summary

This paper introduces a claim-centric auditing framework for identifying error spans in deep-research agent trajectories, along with a new benchmark TELBench, improving process-level reliability assessment.

Deep-research agents solve tasks through long trajectories of search, tool use, evidence inspection, and answer synthesis. Evaluation based on final answers shows whether an agent succeeds, but not which parts of the trajectory make the answer unreliable. We study span-level error localization for deep-research agents. We collect 2,790 real trajectories from two agent frameworks, three backbone models, and three benchmarks, convert raw logs into semantic spans, and annotate harmful error spans through LLM-assisted expert review. From these annotations, we build TELBench, a 1,000-instance benchmark for identifying error spans among normal exploration, failed searches, tentative hypotheses, and harmless noise. We further propose DRIFT, a claim-centric auditing framework that tracks agent claims, checks their support in trajectory evidence, and marks spans where unsupported or conflicting claims affect the answer path. Experiments across model families and auditing frameworks show that DRIFT improves span-level error localization and first-error accuracy by up to 30 percentage points. Our work provides a process-level view of reliability in deep-research agents.
Original Article
View Cached Full Text

Cached at: 06/04/26, 03:41 AM

Paper page - Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories

Source: https://huggingface.co/papers/2606.02060

Abstract

Deep-research agents can be audited using a claim-centric framework that identifies error spans in their reasoning trajectories, improving reliability assessment beyond just final answer evaluation.

Deep-research agentssolve tasks through long trajectories of search, tool use, evidence inspection, and answer synthesis. Evaluation based on final answers shows whether an agent succeeds, but not which parts of the trajectory make the answer unreliable. We studyspan-level error localizationfordeep-research agents. We collect 2,790 real trajectories from twoagent frameworks, threebackbone models, and three benchmarks, convert raw logs into semantic spans, and annotate harmfulerror spansthroughLLM-assisted expert review. From these annotations, we buildTELBench, a 1,000-instance benchmark for identifyingerror spansamong normal exploration, failed searches, tentative hypotheses, and harmless noise. We further proposeDRIFT, aclaim-centric auditingframework that tracks agent claims, checks their support intrajectory evidence, and marks spans where unsupported or conflicting claims affect the answer path. Experiments across model families and auditing frameworks show thatDRIFTimprovesspan-level error localizationand first-error accuracy by up to 30 percentage points. Our work provides a process-level view of reliability indeep-research agents.

View arXiv pageView PDFProject pageGitHub4Add to collection

Get this paper in your agent:

hf papers read 2606\.02060

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.02060 in a model README.md to link it from this page.

Datasets citing this paper1

#### NJU-LINK/TELBench Updated27 minutes ago • 49

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.02060 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

SearchAuditor: Auditing and Attributing Failures in Long-Horizon Search Agents

arXiv cs.AI

This paper introduces SearchAuditBench, a benchmark of 1,243 failed long-horizon search-agent trajectories with expert annotations, and SearchAuditor, a multi-perspective auditing framework that localizes, attributes, and repairs agent failures. Experiments show SearchAuditor outperforms baselines, achieving a 32.3% end-to-end pass rate with frontier models like GPT-5.5.

Long-Horizon Agent Trajectory Attribution: A Unified Benchmark and Fine-Grained Annotation Framework

arXiv cs.AI

This paper introduces a unified benchmark and fine-grained annotation framework for long-horizon agent trajectory attribution, enabling evaluation of primary attribution localization and attribution-chain recovery across diverse settings. It provides over 1,300 annotated trajectories from existing agent benchmarks and releases a reusable annotation skill for standardizing future trajectories.

AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systems

arXiv cs.CL

This paper introduces AgentForesight, a framework for online auditing and early failure prediction in LLM-based multi-agent systems. It presents a new dataset, AFTraj-22K, and a specialized model, AgentForesight-7B, which outperforms leading proprietary models in detecting decisive errors during trajectory execution.