DR^{3}-Eval: Towards Realistic and Reproducible Deep Research Evaluation
Summary
DR³-Eval is a benchmark for evaluating deep research agents on multimodal, multi-file report generation with a realistic web environment simulation and comprehensive evaluation framework measuring information recall, factual accuracy, citation coverage, instruction following, and depth quality.
View Cached Full Text
Cached at: 04/20/26, 08:29 AM
Paper page - DR^{3}-Eval: Towards Realistic and Reproducible Deep Research Evaluation
Source: https://huggingface.co/papers/2604.14683 Published on Apr 16
#2 Paper of the day (https://huggingface.co/papers/date/2026-04-17) Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
DR^\{3\}-Eval is a benchmark for evaluating deep research agents on multimodal, multi-file report generation, featuring a realistic simulation of web environments and a comprehensive evaluation framework.
Deep Research Agents (https://huggingface.co/papers?q=Deep%20Research%20Agents)(DRAs) aim to solve complex, long-horizon research tasks involving planning, retrieval,multimodal understanding (https://huggingface.co/papers?q=multimodal%20understanding), andreport generation (https://huggingface.co/papers?q=report%20generation), yet their evaluation remains challenging due to dynamic web environments and ambiguous task definitions. We propose DR^{3}-Eval, a realistic and reproducible benchmark for evaluatingdeep research agents (https://huggingface.co/papers?q=deep%20research%20agents)on multimodal, multi-filereport generation (https://huggingface.co/papers?q=report%20generation). DR^{3}-Eval is constructed from authentic user-provided materials and paired with a per-task staticresearch sandbox corpus (https://huggingface.co/papers?q=research%20sandbox%20corpus)that simulates open-web complexity while remaining fully verifiable, containing supportive documents, distractors, and noise. Moreover, we introduce amulti-dimensional evaluation framework (https://huggingface.co/papers?q=multi-dimensional%20evaluation%20framework)measuringInformation Recall (https://huggingface.co/papers?q=Information%20Recall),Factual Accuracy (https://huggingface.co/papers?q=Factual%20Accuracy),Citation Coverage (https://huggingface.co/papers?q=Citation%20Coverage),Instruction Following (https://huggingface.co/papers?q=Instruction%20Following), andDepth Quality (https://huggingface.co/papers?q=Depth%20Quality), and validate its alignment with human judgments. Experiments with our developedmulti-agent system (https://huggingface.co/papers?q=multi-agent%20system)DR^{3}-Agent based on multiplestate-of-the-art language models (https://huggingface.co/papers?q=state-of-the-art%20language%20models)demonstrate that DR^{3}-Eval is highly challenging and reveals critical failure modes in retrieval robustness andhallucination control (https://huggingface.co/papers?q=hallucination%20control). Our code and data are publicly available.
View arXiv page (https://arxiv.org/abs/2604.14683)View PDF (https://arxiv.org/pdf/2604.14683)GitHub28 (https://github.com/NJU-LINK/DR3-Eval)Add to collection (https://huggingface.co/login?next=%2Fpapers%2F2604.14683)
Get this paper in your agent:
hf papers read 2604\.14683
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2604.14683 in a model README.md to link it from this page.
Datasets citing this paper1
NJU-LINK/DR3-Eval Viewer• Updated3 days ago • 100 • 1.98k • 1 (https://huggingface.co/datasets/NJU-LINK/DR3-Eval)
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2604.14683 in a Space README.md to link it from this page.
Collections including this paper2
Similar Articles
Mind DeepResearch Technical Report
MindDR is a multi-agent deep research framework using a three-agent architecture (Planning, DeepSearch, Report) and a four-stage training pipeline, achieving competitive performance with ~30B-parameter models on multiple benchmarks. Developed by Li Auto and deployed as an online product, it also introduces MindDR Bench, a 500-query Chinese benchmark for evaluating deep research capabilities.
DailyReport: An Open-ended Benchmark for Evaluating Search Agents on Daily Search Tasks
DailyReport is an open-ended benchmark for evaluating search agents on daily search tasks, featuring 150 tasks and 3,546 rubrics for interpretable, user-centric evaluation.
Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agent
Video-DeepResearch (Video-DR) extends multimodal agents from static images to continuous video streams, introducing a decoupled perception-exploration pipeline and a new benchmark Video-DR-Bench. Their Video-DeepResearch-35B-A3B model achieves 64.0% accuracy, surpassing Claude-4.5-Sonnet, GPT-5, and Gemini 2.5 Pro.
DuMate-DeepResearch: An Auditable Multi-Agent System with Recursive Search and Rubric-Grounded Reasoning
This technical report introduces DuMate-DeepResearch, a multi-agent framework for deep research tasks that decouples the agent core from a tool ecosystem, and incorporates graph-based dynamic planning, recursive two-level execution, and rubric-based test-time optimization. The system achieves state-of-the-art results on two deep research benchmarks, demonstrating the value of auditable agent infrastructure.
Self-Evolving Deep Research via Joint Generation and Evaluation
Researchers from HKUST, ByteDance, and UCL propose SCORE, a co-evolutionary training framework that jointly trains an LLM as both a deep research report generator and an evaluator, using a meta-harness to dynamically adjust evaluation difficulty and prevent reward saturation. Experiments show consistent improvement in open-ended research report quality.