Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents

# A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents
Source: [https://arxiv.org/html/2609.11318](https://arxiv.org/html/2609.11318)
Minghao Guo\*minghaoguo20@gmail\.comMeng Cao\*mengcaopku@gmail\.comAffiliation:Mohamed bin Zayed University of Artificial IntelligenceSui Zhaosui@mail\.ustc\.edu\.cnAffiliation:University of Science and Technology of ChinaSiyu Ningningsiyu@mail\.ustc\.edu\.cnAffiliation:University of Science and Technology of ChinaXin Wangwangxin2252@gmail\.comAffiliation:University of Science and Technology of ChinaHaoze ZhaoHaoze\.Zhao@mbzuai\.ac\.aeAffiliation:Mohamed bin Zayed University of Artificial IntelligenceJiaxuan Yangyangjiaxuan@zju\.edu\.cnAffiliation:Zhejiang UniversityHaihong Haohaohaihong@mail\.ustc\.edu\.cnAffiliation:University of Science and Technology of ChinaMingfei HanMingfei\.Han@mbzuai\.ac\.aeAffiliation:Mohamed bin Zayed University of Artificial IntelligenceShunlin RongAffiliation:TencentHaijun WuAffiliation:TencentXiaodan LiangXiaodan\.Liang@mbzuai\.ac\.aeAffiliation:Mohamed bin Zayed University of Artificial IntelligenceXiaojun ChangXiaojun\.Chang@mbzuai\.ac\.aeAffiliation:Mohamed bin Zayed University of Artificial Intelligence

###### Abstract

Deep research agents are increasingly capable of searching the web, invoking tools, examining multimodal evidence, and synthesizing information from multiple sources\. Yet most of existing benchmarks are confined to relatively medium\-horizon evidence exploration\. MM\-BrowseComp, for example, contains an average of only 3\.0 checklist items per question, leaving the ability of agents to sustain long\-horizon deep research underexplored\. To this end, we introduce Mr\.LHDR \(Multimodalreal\-worldLong\-HorizonDeepResearch\), a benchmark designed to evaluate real\-world deep research over long, irreducible chains of interdependent evidence across eight categories\. Each question is constructed from a hidden Node\-Relation graph and requires, on average, 12\.1 necessary intermediate conclusions with a mean dependency depth of 10\.4 before reaching a short, unique, and verifiable answer\. We design each question so that multiple key steps involve images, maps, PDFs, logos, charts, tables, or video frames\. As a hard inclusion criterion, every question contains at least one piece of non\-text evidence that changes the reasoning state\. Mr\.LHDR therefore evaluates both whether an agent returns the final answer and whether its stated intermediate conclusions are correct given the annotated dependencies\. We evaluate regular models, dedicated deep research systems, and framework\-based agents using Overall Accuracy \(OA\), Strict Accuracy \(SA\), Checklist Score \(CS\), and Dependency\-Aware Checklist Score \(DACS\)\. Empirically, we find that: \(1\) the run with the highest OA, GPT\-5\.5, reaches only 43\.1% OA and 34\.3% SA, while the dedicated deep research system with the highest OA, o3 Deep Research, reaches 32\.4% OA and 19\.6% SA, showing that final\-answer accuracy substantially overstates complete, dependency\-consistent task success; \(2\) withholding images lowers DACS by 12\.6 points in a controlled ablation, confirming that non\-text evidence materially contributes to the reasoning chain; and \(3\) across checklist\-length ranges with sufficient observations, SA decreases monotonically with chain length for every representative model\. Together, these findings identify sustained, dependency\-consistent evidence integration, instand of isolated fact retrieval, as a central bottleneck for current deep research agents\. Code and data are available at[https://github\.com/minghaoguo20/Mr\-LHDR\-eval](https://github.com/minghaoguo20/Mr-LHDR-eval)\.

††footnotetext:∗Equal contribution\.## 1Introduction

AI agents are increasingly expected to conduct deep research rather than answer isolated questions\([Zheng et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib37);[Zhang et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib38);[Du et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib19)\)\. A useful research assistant should decompose an underspecified goal, search across heterogeneous sources, inspect visual and document evidence, maintain intermediate conclusions, and verify a final answer against the accumulated evidence\. Many real\-world tasks follow exactly this pattern: an agent may need to identify an object in an image, trace an entity chain across the web, read a map, chart, or PDF, and then combine the evidence into a short factual answer\. The difficulty stems not only from obscure individual facts, but also from the risk that an early error redirects the entire investigation\.

Existing benchmarks have made important progress in evaluating web\-enabled agents\([Wei et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib7);[Zhou et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib18);[Li et al\., 2025d](https://arxiv.org/html/2609.11318#bib.bib20);[Du et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib19)\), but they do not fully capture this setting\. BrowseComp\-style tasks emphasize hard\-to\-find information on the open web and are useful for measuring search and browsing competence, but they remain primarily text\-centered\. MM\-BrowseComp\-style tasks add multimodal browsing, but remain concentrated on short\- to medium\-horizon research: MM\-BrowseComp contains only 3\.0 checklist items per question on average\([Li et al\., 2025d](https://arxiv.org/html/2609.11318#bib.bib20)\), leaving long chains of dependent evidence steps largely untested\. Mr\.LHDR targets the long, tightly coupled, multi\-source, cross\-modal research processes found in real\-world investigations\. Strong agents may solve individual browsing or multimodal steps, yet still fail when a task requires many dependent steps, cross\-modal bridges, and repeated verification\. Figure[1](https://arxiv.org/html/2609.11318#S1.F1)illustrates this distinction through one public MM\-BrowseComp example and one Mr\.LHDR example\.

![Refer to caption](https://arxiv.org/html/2609.11318v1/lhdr_contrast.png)Figure 1:Comparison of annotated structural horizon\. The public MM\-BrowseComp image\-input example uses four sequential checklist items; its original 224\-item release averages 3\.0 items per question\([Li et al\., 2025d](https://arxiv.org/html/2609.11318#bib.bib20)\)\. The Mr\.LHDR example uses 12 necessary conclusions in two prerequisite branches, while the full benchmark averages 12\.1 conclusions and a dependency depth of 10\.4\. Both settings require multimodal web research and short verifiable answers\. Nodes denote conclusions rather than browsing actions; arrows denote annotated prerequisites\.We introduce Mr\.LHDR \(Multimodal real\-world Long\-Horizon Deep Research\), a benchmark for real\-world open\-web research tasks\. Mr\.LHDR contains 102 questions across eight categories\. Each item includes a natural\-language question, a short verifiable answer, source evidence, multimodal evidence, an irreducible checklist of necessary intermediate conclusions, a category label, a coarse authored\-template label, and operation tags\. The final answers are intentionally short, unique, and stable, such as a person name, place name, organization name, year, number, color, or short phrase\. Mr\.LHDR frames real\-world research tasks as verifiable short\-answer questions whose difficulty arises from real evidence dependencies\.

The design of Mr\.LHDR follows three principles\.\(1\) Structural horizon\.Long\-horizon denotes the depth of prerequisite\-linked evidence structures rather than input or output length: an agent must preserve an evolving research state across dependent steps, not merely process a long prompt or produce a long response\.\(2\) Open\-world multimodal investigation\.Solving an item may require decomposing the goal, locating evidence across webpages, PDFs, images, maps, videos, audio, and tables, bridging information across modalities, preserving intermediate conclusions, repeatedly verifying constraints, and returning a final short answer\.\(3\) Dependency\-aware evaluation\.Mr\.LHDR reports Overall Accuracy \(OA\), Strict Accuracy \(SA\), Checklist Score \(CS\), and Dependency\-Aware Checklist Score \(DACS\) to distinguish final\-answer correctness, complete response correctness, intermediate\-conclusion coverage, and dependency\-consistent coverage\. SA, CS, and DACS are computed from conclusions stated in the submitted response, with DACS additionally requiring their annotated prerequisites to be satisfied\. Because these metrics neither observe the agent’s internal reasoning nor establish that a stated conclusion came from actual retrieval, they evaluate dependency\-consistent stated conclusions rather than actual evidence\-chain completion\.

We evaluate 25 regular models, dedicated deep research systems, and framework\-based agents on Mr\.LHDR, yielding four main findings\.First, current systems remain far from saturation\.The regular\-model run with the highest OA, GPT\-5\.5, achieves only 43\.1% OA and 34\.3% SA; the strongest dedicated system by OA, o3 Deep Research, reaches 32\.4% OA and 19\.6% SA; and DeerFlow \(qwen3\-vl\-235b\) reaches 15\.7% OA and 9\.8% SA\.Second, final\-answer success overstates complete task execution\.The substantial OA–SA gaps show that a correct final answer often coexists with missing required conclusions, while the lower DACS than CS reveals conclusions whose annotated prerequisites are absent\.Third, visual evidence materially improves evidence\-chain recovery\.In a controlled same\-model ablation, providing images raises DACS from 21\.6% to 34\.2%, a gain of 12\.6 points \(95% CI \[6\.9, 18\.9\]\)\.Fourth, longer evidence chains increase strict\-completion pressure\.Across the three checklist\-length ranges with at least 20 items, SA decreases monotonically with checklist length for every representative model\. Because confidence intervals for many neighboring systems overlap, these results support broad capability patterns rather than a statistically separable ranking\.

##### Contributions\.

We make the following contributions:

- •A benchmark for long\-horizon multimodal deep research\.We introduce Mr\.LHDR, an open\-web benchmark comprising 102 questions across eight real\-world categories\. Each question is constructed using a Node\-Relation design and paired with multimodal source evidence, a short, unique, and verifiable answer, and an irreducible checklist structured as an author\-verified dependency DAG\. The benchmark contains 1,231 annotated intermediate conclusions with a mean dependency depth of 10\.4, operationalizing long\-horizon difficulty through prerequisite\-linked evidence structures\.
- •A dependency\-aware evaluation protocol\.We introduce Dependency\-Aware Checklist Score \(DACS\), which recursively conditions the credit for each stated conclusion on the correctness of its annotated prerequisites\. Together with Overall Accuracy \(OA\), Strict Accuracy \(SA\), and Checklist Score \(CS\), the protocol distinguishes among final\-answer correctness, complete response correctness, intermediate\-conclusion coverage, and dependency\-consistent coverage, enabling uniform evaluation across heterogeneous research systems\.
- •A systematic empirical study of current research systems\.We evaluate 25 systems spanning tool\-augmented VLMs, tool\-free VLMs, dedicated deep research systems, and framework\-based agents\. The results reveal systematic gaps between final\-answer correctness and complete stated\-conclusion correctness, as well as between checklist coverage and dependency\-consistent coverage\. A controlled image ablation using the same model further demonstrates the contribution of visual evidence to recovering dependency\-constrained conclusions\. Bootstrap uncertainty estimation and robustness analyses of the judge and dependency graph support these findings\.

## 2Related Work

##### Multimodal Understanding and Evidence Benchmarks\.

Recent multimodal large language models have moved beyond image–text perception to support documents, charts, videos, screens, long contexts, and agent\-centric visual reasoning\.[Li et al\. \(2025a\)](https://arxiv.org/html/2609.11318#bib.bib22);[Guo et al\. \(2025\)](https://arxiv.org/html/2609.11318#bib.bib23);[Team et al\. \(2025\)](https://arxiv.org/html/2609.11318#bib.bib24);[Bai et al\. \(2025\)](https://arxiv.org/html/2609.11318#bib.bib25)exemplify this shift toward general\-purpose multimodal understanding, long\-context grounding, and reasoning over interleaved visual and textual inputs\. Multimodal evaluation has evolved accordingly\.[Liu et al\. \(2024\)](https://arxiv.org/html/2609.11318#bib.bib8)and[Yue et al\. \(2023\)](https://arxiv.org/html/2609.11318#bib.bib27)assess broad and expert\-level reasoning;[Masry et al\. \(2022\)](https://arxiv.org/html/2609.11318#bib.bib28)and[Mathew et al\. \(2020\)](https://arxiv.org/html/2609.11318#bib.bib29)focus on charts and document images; and newer evidence\-oriented benchmarks examine long multimodal documents, retrieval\-augmented visual culture understanding, multimodal document RAG, and conflicts between parametric knowledge and external multimodal evidence\.\([Duan et al\., 2024](https://arxiv.org/html/2609.11318#bib.bib26);[Huybrechts et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib40);[Li et al\., 2025b](https://arxiv.org/html/2609.11318#bib.bib30);[Dong et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib31);[Jia et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib32)\)Collectively, these studies establish the importance of evidence selection, cross\-modal integration, domain knowledge, and robustness to conflicting priors\. In these benchmarks, however, evidence is typically supplied with the query, contained in a fixed corpus, or retrieved within a predefined document or RAG setting\. Mr\.LHDR considers a complementary open\-research setting in which an agent must discover necessary non\-textual evidence on the open web, connect it to evolving textual and symbolic constraints, and retain the resulting intermediate conclusions throughout a long\-horizon Node\-Relation research process\.

##### Tool\-Enhanced Browsing and Deep Research Agents\.

Tool\-enhanced agents extend language and multimodal models with search engines, browsers, APIs, code execution, and other external tools\.[Nakano et al\. \(2021\)](https://arxiv.org/html/2609.11318#bib.bib33),[Yao et al\. \(2023\)](https://arxiv.org/html/2609.11318#bib.bib1)and[Schick et al\. \(2023\)](https://arxiv.org/html/2609.11318#bib.bib34)established early paradigms for browser\-assisted question answering, interleaved reasoning and acting, and learned tool invocation\. More recent work on search and deep\-research training has shifted from retrieval\-augmented answering toward agents that plan searches, decompose questions, inspect sources, and revise intermediate states over multiple turns\. Examples include[Li et al\. \(2025e\)](https://arxiv.org/html/2609.11318#bib.bib35);[Song et al\. \(2025\)](https://arxiv.org/html/2609.11318#bib.bib36);[Jin et al\. \(2025\)](https://arxiv.org/html/2609.11318#bib.bib9);[Zheng et al\. \(2025\)](https://arxiv.org/html/2609.11318#bib.bib37);[Zhang et al\. \(2025\)](https://arxiv.org/html/2609.11318#bib.bib38);[Li et al\. \(2025c\)](https://arxiv.org/html/2609.11318#bib.bib39)User\-facing systems, including[OpenAI \(2025\)](https://arxiv.org/html/2609.11318#bib.bib10);[Google \(2024\)](https://arxiv.org/html/2609.11318#bib.bib11);[Perplexity AI \(2025\)](https://arxiv.org/html/2609.11318#bib.bib12);[xAI \(2025\)](https://arxiv.org/html/2609.11318#bib.bib13);[Microsoft \(2024\)](https://arxiv.org/html/2609.11318#bib.bib14), and open\-source frameworks such as[ByteDance \(2026\)](https://arxiv.org/html/2609.11318#bib.bib15)reflect the same shift toward multi\-step research workflows and report synthesis\. Although these developments show that agents can search, browse, invoke tools, collect sources, and synthesize long\-form outputs, they leave a more specific evaluation question open: can an agent maintain the research state itself—including the objective, entities, constraints, source support, and multimodal bridge conclusions—across many dependent steps? This question is the focus of Mr\.LHDR\.

##### Browsing, Search, and Deep Research Benchmarks\.

Existing benchmarks evaluate related agent capabilities from several perspectives\.[Yao et al\. \(2022\)](https://arxiv.org/html/2609.11318#bib.bib16);[Deng et al\. \(2023\)](https://arxiv.org/html/2609.11318#bib.bib2);[Zhou et al\. \(2024\)](https://arxiv.org/html/2609.11318#bib.bib3);[He et al\. \(2024\)](https://arxiv.org/html/2609.11318#bib.bib4);[Xie et al\. \(2024\)](https://arxiv.org/html/2609.11318#bib.bib5);[Mialon et al\. \(2024\)](https://arxiv.org/html/2609.11318#bib.bib6);[Jiang et al\. \(2024\)](https://arxiv.org/html/2609.11318#bib.bib17)span e\-commerce interaction, real\-website navigation, realistic web environments, multimodal web operation, desktop control, general assistant tasks, and multimodal search\. Information\-seeking evaluations more directly related to deep research include[Wei et al\. \(2025\)](https://arxiv.org/html/2609.11318#bib.bib7)and[Zhou et al\. \(2025\)](https://arxiv.org/html/2609.11318#bib.bib18), which emphasize hard\-to\-locate short answers on the open web;[Li et al\. \(2025d\)](https://arxiv.org/html/2609.11318#bib.bib20), which requires multimodal browsing evidence and provides verified checklists; and[Du et al\. \(2025\)](https://arxiv.org/html/2609.11318#bib.bib19), which evaluates long\-form research reports\. Complementary agentic\-search benchmarks examine dynamic information seeking, source attribution, reproducibility over fixed corpora, process\-aware search, and the breadth or depth of evidence collection\.\([Xi et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib41);[Gou et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib42);[Chen et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib43);[Xu et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib44);[Wong et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib45);[Lan et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib46)\)Recent work narrows the gap further\.[Tao et al\. \(2026\)](https://arxiv.org/html/2609.11318#bib.bib50)emphasizes provenance\-aware browsing from fine\-grained visual cues, while[Zhang et al\. \(2026\)](https://arxiv.org/html/2609.11318#bib.bib47)introduces expert\-validated subgoals and a subgoal\-coverage process score for open\-web multimodal browsing\.[Gupta et al\. \(2026\)](https://arxiv.org/html/2609.11318#bib.bib49)evaluates causal\-chain open\-web research through objectively verifiable single or set\-valued answers, but scores outcomes rather than intermediate chains\.[Ning et al\. \(2026\)](https://arxiv.org/html/2609.11318#bib.bib48)provides step\-wise multimodal reasoning graphs and process metrics, albeit over a fixed local knowledge base\. Report\-oriented benchmarks address a different output regime\.[Huang et al\. \(2026\)](https://arxiv.org/html/2609.11318#bib.bib52)assesses citation\-grounded multimodal research reports, while[Ye et al\. \(2026\)](https://arxiv.org/html/2609.11318#bib.bib51)jointly evaluates report quality, factuality, and exposed research trajectories on text\-only and multimodal tasks\. Mr\.LHDR complements these settings by combining open\-web search, required real\-world multimodal evidence, concise verifiable answers, long\-horizon Node\-Relation structures, and response\-level dependency\-aware checklist evaluation\. Here, horizon refers to maintaining research state across prerequisite\-linked conclusions, rather than producing long\-form output, processing long\-context input, or executing a prescribed number of browsing actions\. Unlike unordered checklist or subgoal coverage, DACS uses an item\-specific Directed Acyclic Graph \(DAG\)\. It withholds downstream credit when required predecessors are not satisfied\. Unlike trace\-based or fixed\-corpus process evaluation, DACS applies to heterogeneous systems using only their submitted responses, without claiming to observe latent reasoning or tool\-use traces\. Table[1](https://arxiv.org/html/2609.11318#S2.T1)compares Mr\.LHDR with representative short\-answer browsing and structured\-search benchmarks in terms of retrieval setting, multimodal requirements, output form, gold intermediate structure, and prerequisite\-gated scoring\.

Table 1:Comparison with representative browsing and structured\-search benchmarks\. “Gold structure” denotes task\-level supervision of intermediate conclusions or retrieval steps; “prerequisite\-gated” indicates that a downstream step receives credit only when its required predecessors are also satisfied\. “Open” denotes the open web and “local KB” a fixed retrieval corpus\.

## 3Mr\.LHDR Benchmark

### 3\.1Task Definition and Annotation Schema

During evaluation, the model sees only the natural\-language question and its associated image evidence; the Node\-Relation graph, checklist, sources, and reference answer remain hidden\.

Mr\.LHDR is a real\-world multimodal deep research benchmark constructed by a six\-person annotation team of master’s\- and PhD\-level AI researchers\. Its data span eight topical categories: media, people, organizations, geography, society, academia, technology, and sports\. Each item also retains one coarse authored\-template label, Linear or Multi\-branch, and one or more operation tags from Constraint, Symbolic, Temporal, and Numerical\. The template label records the intended construction pattern; it does not define the exact topology of the subsequently verified dependency DAG\. Beyond these labels, each item contains a natural\-language question, a short verifiable answer, source evidence, image or other non\-text evidence, and an irreducible checklist of necessary intermediate conclusions\. This schema supports both final\-answer evaluation and post\-hoc response analysis: the answer field supports answer\-level scoring, the checklist supports assessment of whether the submitted response correctly states the necessary conclusions, and the evidence fields support task verification\.

The eight categories cover common real\-world research settings: cultural media, public figures, organizations, geospatial reasoning, social and historical events, academic artifacts, technologies, and sports\. Linear denotes an item authored around a predominantly sequential route, whereas Multi\-branch denotes one intentionally authored around multiple evidence routes whose partial conclusions are later combined\. These labels summarize construction intent\. The verified checklist DAG is the authoritative representation of actual dependencies and may exhibit additional partial independence or local branching under either template\. The operation tags describe local pressures within the chain rather than independent task families\. Constraint marks candidate filtering or uniqueness conditions; Symbolic covers logos, abbreviations, names, colors, symbols, and format conversions; Temporal covers chronological ordering and relative\-time constraints; Numerical covers numbers, rankings, coordinates, statistics, and unit calculations\. Together, these labels support analysis of both an item’s topic and the locations of structural pressure and demanding local operations within its research chain\.

##### Illustrative example\.

The case study in Appendix Table[22](https://arxiv.org/html/2609.11318#A7.T22)provides a compact example\. The intended chain identifies the pictured phone as an Apple iPhone 15 Pro Max, links it to the A17 Pro and the Apple Silicon team’s relationship with Synopsys, and traces the phone’s titanium frame to the A\-12 and SR\-71 aircraft\. The branch corresponding to the later\-retired aircraft proceeds through the SR\-71 Flight Simulator to the Frontiers of Flight Museum, Smithsonian Affiliations, and the Smithsonian Institution\. A U\.S\. Department of State video then supplies the first word, “climate,” on the third interviewee’s shirt\. Initially misidentifying the phone as a Samsung device redirects the processor, aircraft, and museum links; later guesses about Smithsonian\-related nodes cannot repair the unsupported chain\. Without reproducing the full case table, this example shows why early visual grounding and preservation of intermediate entities matter\.

### 3\.2Benchmark Construction

#### 3\.2\.1Node\-Relation Design Principles

Annotators do not begin Mr\.LHDR items by writing a natural\-language question\. Instead, they first construct a Node\-Relation graph\. A node pairs a core entity with searchable and verifiable properties, such as a person photo, landmark image, map location, paper PDF, official document, poster, logo, chart, timestamp, identity, work, affiliation, venue, or founder\. Each item typically begins with five to six information\-rich nodes\. Nodes are selected not merely for searchable names, but for properties that support later verification and cross\-modal bridging\.

A relation turns a verified property of one node into an entry condition for the next\. Relations may take the form of shared attributes, such as the same location, organization, person, event, or work, or uniqueness constraints, such as a relative year, a founder relation, the first host city of an event, the sister city of a birthplace, or an organization indicated by a visual symbol\. A valid relation must be natural and uniquely convergent: removing any key relation should visibly break the chain rather than merely shorten it\.

The long\-horizon difficulty in Mr\.LHDR arises from dependency structure rather than verbose prompts, long outputs, or obscure trivia\. Long\-horizon requires preserving an evolving research state across dependent evidence steps: a conclusion established at one step becomes a constraint or entry condition for later investigation\. Therefore, the final question should not be answerable through a single query, a single webpage, or a direct search for its final wording\. Later steps must depend on earlier intermediate conclusions, requiring the agent to retain the research objective, resolved entities, and accumulated constraints across many hops\. This design motivates checklist\-based response evaluation: final\-answer correctness alone can obscure whether the submitted response correctly states the necessary intermediate conclusions and their prerequisites\.

Multimodal evidence is a required part of the reasoning chain\. The entry node is preferably grounded in a real\-world non\-text modality, such as an image, map, PDF, video screenshot, logo, product image, chart, or table\. Each Mr\.LHDR item must contain non\-text evidence that changes the reasoning state by identifying an entity, constraining candidates, confirming a visual relation, localizing a map object, or extracting information from a document, chart, or table\.

#### 3\.2\.2AI\-Assisted Construction Workflow

![Refer to caption](https://arxiv.org/html/2609.11318v1/ai_assisted_pipeline.png)Figure 2:The overall construction pipeline for Mr\.LHDR\. AI helps explore the candidate design space through Node\-Relation graphs, over\-complete checklists, draft fields, and question polishing\. Human annotators then prune, revise, and validate the candidates, locking only items that pass source, checklist, multimodal, shortcut, and metadata checks\.Figure[2](https://arxiv.org/html/2609.11318#S3.F2)summarizes the AI\-assisted construction workflow\. AI serves as a candidate\-expansion tool rather than an authority\. Annotators first specify the target category, operation tags, and golden\-example style, then use models to propose Node\-Relation graphs, possible multimodal sources, and over\-complete checklists\. Human annotators prune these candidates into irreducible evidence chains, reject any wording or attribute ambiguity that undermines answer uniqueness or source support, and draft the final question, answer, sources, images, and metadata\. Final admission remains under human control through checks of factual support, multimodal necessity, shortcut resistance, label correctness, and benchmark coverage\. Detailed pipeline stages and annotation decisions are described in Appendix[A\.4](https://arxiv.org/html/2609.11318#A1.SS4)\.

#### 3\.2\.3Quality Standards and Validation Protocol

##### Quality criteria\.

Candidate items must jointly satisfy four quality standards\. First, the target answer must be concise, inspectable, unique under open\-web evidence, and stable within the question’s temporal scope\. Second, the checklist must form an irreducible chain of verifiable conclusions rather than browsing actions\. Every key node, bridge relation, checklist entry, and final answer requires inspectable source support, and removing any necessary step must break reliable derivation\. Third, the question must hide intermediate nodes and resist direct\-answer, single\-query, single\-page, and text\-only shortcuts\. Fourth, non\-text evidence counts only if it changes the reasoning state by identifying an entity, constraining candidates, localizing an object, reading a document or chart, or bridging two nodes\.

##### Validation protocol\.

We enforce these standards through a four\-stage validation protocol\.

Phase 1: Pilot and calibration\.Each of the six annotators first constructs a small pilot batch from golden examples\. The core team audits graph naturalness, unique convergence, path leakage, checklist form, multimodal necessity, and answer format, aligning annotators on the boundary of admissible long\-horizon tasks\.

Phase 2: Full\-scale construction and structured secondary audit\.A separate auditor checks every field in a fixed order: whether the hidden chain can be skipped, whether the question reveals the path, whether the checklist is minimal, and whether sources support every key node, relation, conclusion, and answer\. Items with unsupported conclusions, multiple valid answers, or reducible checklists are returned for revision\.

Phase 3: Multimodal necessity and shortcut check\.Auditors verify that at least one key conclusion depends on non\-text evidence and that this evidence participates in the chain rather than serving as decoration\. They then submit the final question verbatim to a web\-enabled model without providing the hidden graph, checklist, sources, or extra prompts\. If a single response recovers the answer or a reproducible direct\-search path, the item is rewritten, constrained further, or rejected\.

Phase 4: Factual and metadata audit\.Before locking an item, the core team verifies every URL and corresponding claim, checks that the chain is not dominated by Wikipedia or a single text\-only page, confirms the presence of inspectable non\-text evidence, and audits category, template, tag, image, and source metadata\. Only items that pass every gate enter evaluation\.

Table 2:Quality gates applied before an item enters the Mr\.LHDR evaluation set\. These gates operationalize answer uniqueness, no path leakage, irreducibility, source support, multimodal necessity, shortcut filtering, and metadata consistency\.

### 3\.3Benchmark Statistics

Mr\.LHDR contains 102 locked evaluation items, each supported by a multi\-step checklist and multiple evidence sources\. The author\-verified checklist graphs exhibit substantial dependency depth, and parallel branches further increase the overall evidential workload\. The benchmark spans eight real\-world research categories, includes both Linear and Multi\-branch authored templates, and covers symbolic, constraint\-based, numerical, and temporal reasoning\. Figure[3](https://arxiv.org/html/2609.11318#S3.F3)summarizes the label distributions\. Detailed checklist, dependency\-depth, source, image, non\-text\-evidence, and split statistics are reported in Appendix[B](https://arxiv.org/html/2609.11318#A2)\.

Figure 3:Label distribution in Mr\.LHDR: categories, Linear/Multi\-branch authored templates, and operation tags\. Operation tags are multi\-label, so their percentages need not sum to 100%\.

## 4Experiments

### 4\.1Experimental Setup

#### 4\.1\.1Evaluation Set and Coverage

All experiments use the fixed Mr\.LHDR evaluation set, which contains 102 benchmark items\. Each item includes a natural\-language question, a short reference answer, source evidence, an image or other non\-text evidence, category and structure labels, operation tags, and an irreducible checklist of necessary intermediate conclusions\. During evaluation, models receive the question and, when available, its associated image evidence\.

The evaluation set covers all eight Mr\.LHDR categories and both coarse authored\-template labels\. It contains 73 Linear\-template items and 29 Multi\-branch\-template items\. All 102 items have checklist annotations, comprising 1,231 checklist steps in total\. These annotations define the denominators used to compute SA, CS, and DACS in the experiments below\.

#### 4\.1\.2Model Groups

We evaluate 25 systems, divided into four groups according to the research interface and tool access used during evaluation\. Tool\-augmented VLMs receive image input and access to provider\-exposed web search, whereas tool\-free VLMs receive image input but no search tool\. The tool\-free label refers to the evaluation configuration, not an inherent limitation of the model\. Deep Research Systems use dedicated research endpoints, while framework\-based agents run through DeerFlow\([ByteDance, 2026](https://arxiv.org/html/2609.11318#bib.bib15)\)\. Each evaluation consists of a single request\. Any tool use or multi\-step behavior occurs within the provider’s service or the framework, rather than in an agent loop that we built\. Table[3](https://arxiv.org/html/2609.11318#S4.T3)summarizes these groups, and Appendix Table[8](https://arxiv.org/html/2609.11318#A3.T8)lists every model, route, and configuration\.

Table 3:Summary of the model groups evaluated in Mr\.LHDR\. The full model configuration is reported in the appendix\.
#### 4\.1\.3Metrics

We evaluate Mr\.LHDR along two complementary dimensions: final\-answer correctness and the dependency consistency of the conclusions stated in a response\. For a fixed model and itemii, letyi∈\{0,1\}y\_\{i\}\\in\\\{0,1\\\}denote the judge’s binary decision on final\-answer correctness\. LetGi=\(Vi,Ei\)G\_\{i\}=\(V\_\{i\},E\_\{i\}\)denote the verified checklist dependency DAG, with itsmi=\|Vi\|m\_\{i\}=\|V\_\{i\}\|nodes indexed byjj\. Finally, letai​j∈\{0,1\}a\_\{ij\}\\in\\\{0,1\\\}indicate whether the response correctly states checklist conclusionjjfor itemii\. Over the fixed set ofn=102n=102items, Overall Accuracy \(OA\), Strict Accuracy \(SA\), and Checklist Score \(CS\) are defined in Equations[1](https://arxiv.org/html/2609.11318#S4.E1)–[3](https://arxiv.org/html/2609.11318#S4.E3):

OA\\displaystyle\\mathrm\{OA\}=1n​∑i=1nyi,\\displaystyle=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}y\_\{i\},\(1\)SA\\displaystyle\\mathrm\{SA\}=1n​∑i=1nyi​∏j=1miai​j,\\displaystyle=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}y\_\{i\}\\prod\_\{j=1\}^\{m\_\{i\}\}a\_\{ij\},\(2\)CS\\displaystyle\\mathrm\{CS\}=1n​∑i=1n1mi​∑j=1miai​j\.\\displaystyle=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\frac\{1\}\{m\_\{i\}\}\\sum\_\{j=1\}^\{m\_\{i\}\}a\_\{ij\}\.\(3\)
Thus, OA measures final\-answer correctness\. SA additionally requires the response to state every annotated necessary conclusion correctly\. CS measures average checklist coverage without accounting for dependencies and is retained as an auxiliary metric\.

Our primary process metric is the Dependency\-Aware Checklist Score \(DACS\)\. LetDi​j⊆ViD\_\{ij\}\\subseteq V\_\{i\}denote the direct prerequisites of nodejjin itemii\. We recursively define its dependency\-grounded creditsi​js\_\{ij\}and the resulting DACS in Equations[4](https://arxiv.org/html/2609.11318#S4.E4)and[5](https://arxiv.org/html/2609.11318#S4.E5), respectively:

si​j\\displaystyle s\_\{ij\}=ai​j​∏k∈Di​jsi​k,\\displaystyle=a\_\{ij\}\\prod\_\{k\\in D\_\{ij\}\}s\_\{ik\},\(4\)DACS\\displaystyle\\mathrm\{DACS\}=1n​∑i=1n1mi​∑j=1misi​j\.\\displaystyle=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\frac\{1\}\{m\_\{i\}\}\\sum\_\{j=1\}^\{m\_\{i\}\}s\_\{ij\}\.\(5\)
where an empty product equals one\. A node therefore receives credit only if both the node itself and all of its transitive prerequisites are satisfied\. This recursion preserves credit for independent branches but withholds credit from a conclusion if any required ancestor is missing\. DACS thus measures the dependency consistency of stated conclusions\.

#### 4\.1\.4LLM Judge

The LLM Judge compares each model response against the reference answer and checklist\. It outputs a binary decision on final\-answer correctness and a checklist completion vector\. OA, SA, CS, and DACS are then computed deterministically from these outputs and the frozen dependency graphs, using the fixed denominator of 102 items\. Failed or missing runs are assigned an incorrect final\-answer decision and an all\-zero checklist vector\. The current evaluation pipeline uses Qwen3\-VL\-235B \(Qwen/Qwen3\-VL\-235B\-A22B\-Instruct\-FP8\) as the judge model\.

### 4\.2Main Results

#### 4\.2\.1Overall Capability

Mr\.LHDR remains challenging even for strong systems\. As shown in Table[4](https://arxiv.org/html/2609.11318#S4.T4), GPT\-5\.5 has the highest OA, with 43\.1% OA, 34\.3% SA, 73\.6% CS, and 70\.3% DACS\. These results remain far from saturation\. Even this run solves only slightly more than two fifths of the benchmark items at the final\-answer level and just over one third under the strict criterion that requires both the answer and all annotated necessary conclusions to be correct\. Among dedicated deep research systems, o3 Deep Research has the highest OA, reaching 32\.4% OA, 19\.6% SA, 56\.1% CS, and 49\.8% DACS\. These values are the highest OA point estimates, not a statistically separable ranking\. With only 102 items, the confidence intervals of adjacent systems overlap substantially, and Appendix[E\.1](https://arxiv.org/html/2609.11318#A5.SS1)shows that most per\-model differences are not statistically separable\. Framework\-based agents are substantially weaker in this evaluation: DeerFlow using Qwen3\-VL\-235B reaches 15\.7% OA, 9\.8% SA, 41\.8% CS, and 38\.5% DACS\.

Table 4:Main Mr\.LHDR results for all 25 systems in the current evaluation scope, grouped by model capability\. Overall columns report final\-answer accuracy \(OA\), strict answer\-and\-process accuracy \(SA\), checklist score \(CS\), and dependency\-aware checklist score \(DACS\)\. Category columns report diagnostic SA point estimates for each topical category\. All metrics are percentages computed using Qwen3\-VL\-235B \(Qwen/Qwen3\-VL\-235B\-A22B\-Instruct\-FP8\) as the judge\.
#### 4\.2\.2Final Answers vs\. Dependency\-Consistent Stated Conclusions

Final\-answer accuracy remains higher than strict answer\-and\-stated\-conclusion correctness\. Figure[4](https://arxiv.org/html/2609.11318#S4.F4)decomposes OA into SA and the residual gap between them\. GPT\-5\.5 drops from 43\.1% OA to 34\.3% SA, a gap of 8\.8 points\. Similarly, o4\-mini drops from 35\.3% OA to 20\.6% SA, a gap of 14\.7 points, and o3 Deep Research drops from 32\.4% OA to 19\.6% SA, a gap of 12\.8 points\. These measurements show that a correct final answer does not guarantee that the response states every necessary intermediate conclusion\. Conversely, a high DACS indicates broad, dependency\-consistent checklist coverage in the response, but does not imply a high OA\. Neither metric alone verifies actual retrieval or source grounding\.

Figure 4:Decomposition of final\-answer accuracy for models with at least 5% OA\. Each bar separates strict answer\-and\-stated\-conclusion correctness \(SA\) from the residual gap between OA and SA\. The residual segment represents correct final answers for which the response does not state every required conclusion\.DACS further reveals dependency inconsistencies among the stated conclusions\. CS measures how many checklist conclusions are stated correctly anywhere in a response, whereas DACS withholds credit from a conclusion if any required ancestor is missing\. The gap is substantial for many systems\. Llama 4 Maverick reaches 46\.7% CS but only 37\.8% DACS; Gemini 2\.5 Flash reaches 65\.0% CS but 58\.0% DACS; and Grok 4\.20 reaches 69\.3% CS but 62\.4% DACS\. These differences identify downstream statements whose prerequisites are absent from the response, but do not establish how those statements were retrieved or grounded\. Appendix[F](https://arxiv.org/html/2609.11318#A6)provides the complete split\-level results and associated small\-sample caveats\.

### 4\.3Analysis and Ablations

#### 4\.3\.1Image Ablation

We test the necessity of multimodal input by asking the same open VLM \(Qwen3\-VL\-235B\) to answer every item twice: once with the image and once with the image withheld\. The question, reference answer, checklist, and judge remain fixed between the two conditions\. Figure[5](https://arxiv.org/html/2609.11318#S4.F5)presents the results\.

The effect of the image is strongest on metrics that measure the evidence chain\. CS rises from 26\.9% to 39\.3% \(gain 12\.4, 95% CI \[6\.9, 18\.3\]\), while DACS rises from 21\.6% to 34\.2% \(gain 12\.6, CI \[6\.9, 18\.9\]\)\. The CS and DACS gains are positive in 100\.0% and 100\.0% of bootstrap resamples, respectively, and neither interval contains zero\. The gain in final\-answer accuracy has the same direction \(7\.8% versus 3\.9%\)\. The fixed evaluation set includes both groups, which avoids post\-hoc selection based on the ablation outcomes; Section[5](https://arxiv.org/html/2609.11318#S5)characterizes the text\-only routes\. Under this probe, visual evidence is therefore necessary for correctly stating a larger portion of the dependency\-constrained chain\.

This direction is stable across judges: under every judge in the pool, OA, CS, and DACS are higher with images, whereas SA is less stable\. Appendix[E\.3](https://arxiv.org/html/2609.11318#A5.SS3)provides the complete self\-judge analysis\.

Figure 5:Multimodal necessity: the same open source VLM \(Qwen3\-VL\-235B\) answering all 102 items*with*vs*without*the image, under the Mr\.LHDR metrics\. Each row is one metric, sorted by the image−\-text gain: the marker is the gain’s point estimate, the horizontal line its 95% bootstrap CI over the 102 items \(B=10000B=10000, seed 20260803\), and marker area is proportional to the with\-image score\. The two right\-hand columns restate the gain and the fraction of bootstrap resamples with a positive gain as exact numbers\. Removing the image drops every metric; the effect is largest and direction\-robust on the process metrics \(CS, DACS\), showing that images help the model state more of the dependency\-constrained chain, not only the answer\.
#### 4\.3\.2Checklist\-Length Pressure

We use checklist length as a proxy for evidential density\. A longer checklist requires an answer to support more annotated conclusions, but does not directly measure browsing cost or the number of actions taken\. We partition the 102 items into five ranges containing 10, 11–12, 13–15, 16–18, and 19\+ statements\. These ranges contain 35, 34, 23, 6, and 4 items, respectively\. Figure[6](https://arxiv.org/html/2609.11318#S4.F6)reports DACS and SA for three representative models across the five ranges\.

Across the three ranges with more observations \(n≥20n\\geq 20\), SA decreases monotonically with checklist length for every model\. DACS is less uniform: GPT\-5\.5 rises from the 10\-step range to the 11–12 range before falling, whereas Gemini 3 Flash and o3 Deep Research decline across the first two ranges\. All three models, however, reach their lowest DACS in the 13–15 range\. The most consistent pattern therefore appears in strict satisfaction: as the number of required conclusions increases, models become progressively less likely to satisfy the entire checklist, even when they retain partial, dependency\-consistent coverage\.

Figure 6:DACS and SA for representative models across checklist\-length ranges\. The final two shaded ranges contain only six and four items, respectively, and are excluded from the main interpretation\.
#### 4\.3\.3High DACS in Tool\-Free Gemini Models

The Gemini models stand out within the tool\-free group\. Gemini 3\.1 Pro records 70\.1% DACS, and every tested Gemini model outperforms all non\-Gemini tool\-free models, whose scores range from 29\.0% to 37\.8%\. All models in this group are evaluated under the same input and tool conditions: they receive the question and its associated image but cannot use open\-web search\. The high Gemini scores therefore cannot be attributed to the tool\-free setting alone\. Moreover, tool\-free does not mean evidence\-free\. The image ablation in Section[4\.3\.1](https://arxiv.org/html/2609.11318#S4.SS3.SSS1)shows that visual evidence improves dependency\-consistent coverage, suggesting that strong multimodal understanding contributes to Gemini’s results\.

The score distribution provides a more direct explanation\. First, Gemini 3\.1 Pro has both a high CS and a DACS of 70\.1%, with only a small gap between the two metrics\. Many of the checklist conclusions that it states correctly therefore also satisfy their prerequisite dependencies\. Second, its DACS is zero on only 12\.7% of items, while it achieves complete dependency\-consistent coverage on 33\.3%\. DACS awards partial credit for correctly recovered portions of a dependency graph and averages this credit across items\. A model can therefore attain a high DACS by covering substantial, dependency\-consistent portions of many items, even if it does not complete every item\. Gemini’s high score reflects this combination of broad coverage and consistent intermediate conclusions\.

However, Gemini 3\.1 Pro still has low OA and SA, and its rate of complete dependency\-consistent coverage is only 33\.3%\. It often recovers many intermediate conclusions but misses one or more links required to complete the evidence chain and support the final answer\. The metrics capture this distinction: DACS measures dependency\-consistent progress within the chain, whereas OA and SA measure end\-to\-end task completion\. This pattern is also stable across judges\. Under each of the 12 independent judges, at least one Gemini model ranks among the top 2 models by DACS\.

#### 4\.3\.4Failure\-Mode Analysis

To characterize unsupported conclusions beyond the aggregate process scores, we assign each failed checklist statement a diagnostic label from the MM\-BrowseComp taxonomy\([Li et al\., 2025d](https://arxiv.org/html/2609.11318#bib.bib20)\)\. For all 25 systems, the failure set is defined consistently using frozen verdicts from Qwen3\.5\-Flash, which differs from the current judge used for the main metrics\. Qwen3\.6 Plus then assigns the diagnostic labels\. The analysis covers 30,775 checklist\-step decisions from 2,550 model–item runs, of which 17,409 receive failure labels\. Appendix[F\.6](https://arxiv.org/html/2609.11318#A6.SS6)provides the taxonomy, coverage details, and complete counts\.

##### Failure\-label distribution\.

Figure[7](https://arxiv.org/html/2609.11318#S4.F7)reveals two broad patterns\. First, failed steps are concentrated in cases where models override task evidence with prior knowledge or draw conclusions that their reasoning does not adequately support\. This pattern indicates persistent weaknesses in evidence integration and reasoning fidelity\. Second, failure profiles vary with system design\. Knowledge override dominates the failures of tool\-augmented VLMs and Deep Research Systems, whereas framework\-based agents exhibit a broader mixture that also includes tool\-execution and uncategorized failures\. This contrast suggests that retrieval and tool access alone do not ensure evidence\-grounded conclusions, while framework\-based pipelines face a wider range of operational failure surfaces\. Because these distributions are conditioned on failed checklist items and the labels are textual diagnostics, they characterize the composition of observed failures, not group failure rates or the causal effects of tool access, retrieval, or orchestration\.

Figure 7:Distribution of classifier\-assigned failure labels among failed checklist items, aggregated by model group\. Passed checklist items are excluded, and slices below 8% are left unlabeled for readability\.
##### Failure rate by dependency depth\.

We locate unsupported conclusions within the annotated dependency structure rather than by their order in the serialized checklist\. A root node has a dependency depth of zero; for every other node, the depth is one plus the maximum depth of its direct prerequisites\. We normalize depth by the number of levels in each item and partition the nodes into the first, middle, and final thirds\. Across all checklist\-step decisions, the failure rate rises from 48\.8% in the first third to 55\.3% in the middle third and 69\.3% in the final third\. The same monotonic pattern holds within each of the four system groups\. Unsupported conclusions are therefore increasingly prevalent among nodes that lie deeper in the prerequisite structure\. This result is a structural diagnostic, not a temporal trace of model behavior: dependency depth does not reveal when a conclusion was attempted, and the observed gradient does not establish that upstream failures caused downstream failures\.

## 5Limitations and Broader Impact

##### Limitations\.

Our agent\-framework group is instantiated using a single open\-source framework; its results therefore characterize this particular harness—including its planning loop, tool interface, and error handling—rather than agent frameworks in general\. Framework runs may also fail for reasons unrelated to reasoning, such as tool\-execution errors, whereas the metrics score submitted responses without distinguishing harness failures from reasoning failures\. The evidence thus supports only the configuration\-specific finding that this framework did not translate its additional machinery into higher process scores\. Broader conclusions require instrumented tool\-execution rates and a controlled comparison of the same underlying model with and without the framework\.

##### Broader impact\.

Mr\.LHDR rewards responses that state necessary intermediate conclusions rather than providing only plausible final answers\. This encourages more auditable research assistants and reveals failures obscured by final\-answer accuracy, although the resulting process scores do not by themselves verify retrieval provenance\. At the same time, linking evidence across sources and modalities is a dual\-use capability that could be directed at private individuals\. We therefore construct items around public works, organizations, and documented public figures, and do not release private personal data\. Publishing questions that rely on the live web also creates risks of benchmark contamination and automated load on third\-party sites; we withhold hidden reasoning paths and full source chains, encourage the reuse of cached responses, and recommend respecting robots directives and rate limits\.

## 6Conclusion

We introduced Mr\.LHDR, a long\-horizon multimodal deep research benchmark that evaluates submitted responses to real\-world research tasks requiring sustained evidence chains\. Its 102 items are constructed from hidden Node\-Relation graphs, grounded in multi\-source and non\-text evidence, and annotated with irreducible checklists of necessary intermediate conclusions\. This design evaluates not only short\-answer correctness but also whether a submitted response correctly states the necessary conclusions under their annotated dependencies\.

Our evaluation suggests that responses submitted by current systems struggle to preserve dependency\-consistent conclusions\. GPT\-5\.5 attains the highest OA at 43\.1%, but only 34\.3% SA\. The dedicated deep research system with the highest OA, o3 Deep Research, reaches 32\.4% OA and 19\.6% SA\. With 102 items, the wide confidence intervals around these point estimates preclude a complete ranking of systems\. Across most systems, OA exceeds SA and CS exceeds DACS\. This pattern indicates that a response can provide a correct final answer or isolated downstream facts without correctly stating all annotated prerequisites\. Breakdowns by category, structure, checklist length, operation tag, and source density, together with the failure taxonomy, provide exploratory diagnostics rather than robust group\-level conclusions\. Taken together, these response\-level patterns suggest difficulty keeping entities, constraints, and multimodal bridge conclusions aligned across a long dependency chain\.

These findings suggest that deep research agents should be evaluated on both final\-answer correctness and response\-level dependency consistency\. DACS measures whether conclusions are correctly stated under annotated dependencies; it does not prove that those conclusions came from retrieved evidence\. Future evaluation should therefore incorporate citation\- or trace\-grounded verification\. Future systems likewise need stronger research\-state tracking, source and modality binding, branch\-specific prerequisite preservation, and chain\-level verification\.

## References

- Baiet al\.\(2025\)S\. Bai, Y\. Cai, R\. Chen, K\. Chen, X\. Chen, Z\. Cheng, L\. Deng, W\. Ding, C\. Gao, C\. Ge, W\. Ge, Z\. Guo, Q\. Huang, J\. Huang, F\. Huang, B\. Hui, S\. Jiang, Z\. Li, M\. Li, M\. Li, K\. Li, Z\. Lin, J\. Lin, X\. Liu, J\. Liu, C\. Liu, Y\. Liu, D\. Liu, S\. Liu, D\. Lu, R\. Luo, C\. Lv, R\. Men, L\. Meng, X\. Ren, X\. Ren, S\. Song, Y\. Sun, J\. Tang, J\. Tu, J\. Wan, P\. Wang, P\. Wang, Q\. Wang, Y\. Wang, T\. Xie, Y\. Xu, H\. Xu, J\. Xu, Z\. Yang, M\. Yang, J\. Yang, A\. Yang, B\. Yu, F\. Zhang, H\. Zhang, X\. Zhang, B\. Zheng, H\. Zhong, J\. Zhou, F\. Zhou, J\. Zhou, Y\. Zhu, and K\. ZhuQwen3\-vl technical report\.arXiv preprint arXiv:2511\.21631\.Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px1.p1.1)\.
- ByteDance \(2026\)ByteDanceDeerFlow\.External Links:[Link](https://github.com/bytedance/deer-flow)Cited by:[§C\.1](https://arxiv.org/html/2609.11318#A3.SS1.p1.1),[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px2.p1.1),[§4\.1\.2](https://arxiv.org/html/2609.11318#S4.SS1.SSS2.p1.1)\.
- Chenet al\.\(2025\)Z\. Chen, X\. Ma, S\. Zhuang, P\. Nie, K\. Zou, A\. Liu, J\. Green, K\. Patel, R\. Meng, M\. Su, S\. Sharifymoghaddam, Y\. Li, H\. Hong, X\. Shi, X\. Liu, N\. Thakur, C\. Zhang, L\. Gao, W\. Chen, and J\. LinBrowseComp\-plus: a more fair and transparent evaluation benchmark of deep\-research agent\.External Links:2508\.06600,[Link](https://arxiv.org/abs/2508.06600)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- Denget al\.\(2023\)X\. Deng, Y\. Gu, B\. Zheng, S\. Chen, S\. Stevens, B\. Wang, H\. Sun, and Y\. SuMind2Web: towards a generalist agent for the web\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://arxiv.org/abs/2306.06070)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- Donget al\.\(2025\)K\. Dong, Y\. Chang, S\. Huang, Y\. Wang, R\. Tang, and Y\. LiuBenchmarking retrieval\-augmented multimomal generation for document question answering\.ArXivabs/2505\.16470\.External Links:[Link](https://api.semanticscholar.org/CorpusID:278788810)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px1.p1.1)\.
- Duet al\.\(2025\)M\. Du, B\. Xu, C\. Zhu, X\. Wang, and Z\. MaoDeepResearch bench: a comprehensive benchmark for deep research agents\.External Links:2506\.11763,[Link](https://arxiv.org/abs/2506.11763)Cited by:[§1](https://arxiv.org/html/2609.11318#S1.p1.1),[§1](https://arxiv.org/html/2609.11318#S1.p2.1),[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- Duanet al\.\(2024\)H\. Duan, J\. Yang, Y\. Qiao, X\. Fang, L\. Chen, Y\. Liu, X\. Dong, Y\. Zang, P\. Zhang, J\. Wang,et al\.Vlmevalkit: an open\-source toolkit for evaluating large multi\-modality models\.InProceedings of the 32nd ACM International Conference on Multimedia,pp\. 11198–11201\.Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px1.p1.1)\.
- Google \(2024\)GoogleGemini deep research\.External Links:[Link](https://gemini.google/overview/deep-research/)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px2.p1.1)\.
- Gouet al\.\(2025\)B\. Gou, Z\. Huang, Y\. Ning, Y\. Gu, M\. Lin, W\. Qi, A\. Kopanev, B\. Yu, B\. J\. Gutiérrez, Y\. Shu, C\. H\. Song, J\. Wu, S\. Chen, H\. N\. Moussa, T\. Zhang, J\. Xie, Y\. Li, T\. Xue, Z\. Liao, K\. Zhang, B\. Zheng, Z\. Cai, V\. Rozgic, M\. Ziyadi, H\. Sun, and Y\. SuMind2Web 2: evaluating agentic search with agent\-as\-a\-judge\.External Links:2506\.21506,[Link](https://arxiv.org/abs/2506.21506)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- Guoet al\.\(2025\)D\. Guo, F\. Wu, F\. Zhu, F\. Leng, G\. Shi, H\. Chen, H\. Fan, J\. Wang, J\. Jiang, J\. Wang,et al\.Seed1\. 5\-vl technical report\.arXiv preprint arXiv:2505\.07062\.Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px1.p1.1)\.
- Guptaet al\.\(2026\)N\. Gupta, R\. Chatterjee, L\. Haas, C\. Tao, A\. Wang, C\. Liu, H\. Oiwa, E\. Gribovskaya, J\. Ackermann, J\. Blitzer, S\. Goldshtein, and D\. DasDeepSearchQA: bridging the comprehensiveness gap for deep research agents\.External Links:2601\.20975,[Link](https://arxiv.org/abs/2601.20975)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2609.11318#S2.T1.7.1.6.1)\.
- Heet al\.\(2024\)H\. He, W\. Yao, K\. Ma, W\. Yu, Y\. Dai, H\. Zhang, Z\. Lan, and D\. YuWebVoyager: building an end\-to\-end web agent with large multimodal models\.InAnnual Meeting of the Association for Computational Linguistics,External Links:[Link](https://aclanthology.org/2024.acl-long.371/)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- Huanget al\.\(2026\)P\. Huang, Z\. Zhong, Z\. Wan, D\. Zhou, S\. Alam, X\. Wang, Z\. Li, Z\. Dou, L\. Zhu, J\. Xiong, C\. Tao, Y\. Xu, D\. Dimitriadis, T\. Zhang, and M\. ZhangMMDeepResearch\-bench: a benchmark for multimodal deep research agents\.External Links:2601\.12346,[Link](https://arxiv.org/abs/2601.12346)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- Huybrechtset al\.\(2025\)G\. Huybrechts, S\. Ronanki, S\. M\. Jayanthi, J\. G\. M\. Fitzgerald, and S\. R\. VeeravanallurDocument haystack: a long context multimodal image/document understanding vision llm benchmark\.2025 IEEE/CVF International Conference on Computer Vision Workshops \(ICCVW\),pp\. 4121–4129\.External Links:[Link](https://api.semanticscholar.org/CorpusID:280166930)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px1.p1.1)\.
- Jiaet al\.\(2025\)Y\. Jia, K\. Jiang, Y\. Liang, Q\. Ren, Y\. Xin, R\. Yang, F\. Feng, M\. Chen, H\. Lu, H\. Wang, X\. Qu, D\. Liu, L\. Cui, and Y\. DuBenchmarking multimodal knowledge conflict for large multimodal models\.ArXivabs/2505\.19509\.External Links:[Link](https://api.semanticscholar.org/CorpusID:278904487)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px1.p1.1)\.
- Jianget al\.\(2024\)D\. Jiang, R\. Zhang, Z\. Guo, Y\. Wu, J\. Lei, P\. Qiu, P\. Lu, Z\. Chen, C\. Fu, G\. Song, P\. Gao, Y\. Liu, C\. Li, and H\. LiMMSearch: benchmarking the potential of large models as multi\-modal search engines\.External Links:2409\.12959,[Link](https://arxiv.org/abs/2409.12959)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- Jinet al\.\(2025\)B\. Jin, H\. Zeng, Z\. Yue, J\. Yoon, S\. Arik, D\. Wang, H\. Zamani, and J\. HanSearch\-r1: training llms to reason and leverage search engines with reinforcement learning\.External Links:2503\.09516,[Link](https://arxiv.org/abs/2503.09516)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px2.p1.1)\.
- Lanet al\.\(2025\)T\. Lan, B\. Zhu, Q\. Jia, J\. Ren, H\. Li, L\. Wang, Z\. Xu, W\. Luo, and K\. ZhangDeepWideSearch: benchmarking depth and width in agentic information seeking\.External Links:2510\.20168,[Link](https://arxiv.org/abs/2510.20168)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- Liet al\.\(2025a\)B\. Li, Y\. Zhang, D\. Guo, R\. Zhang, F\. Li, H\. Zhang, K\. Zhang, P\. Zhang, Y\. Li, Z\. Liu, and C\. LiLLaVA\-onevision: easy visual task transfer\.Transactions on Machine Learning Research\.Note:External Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=zKv8qULV6n)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px1.p1.1)\.
- Liet al\.\(2025b\)J\. Li, Y\. Yuan, W\. Li, M\. Aliannejadi, D\. Hershcovich, A\. Søgaard, I\. Vuli’c, W\. Zhang, P\. P\. Liang, Y\. Deng, and S\. J\. BelongieRAVENEA: a benchmark for multimodal retrieval\-augmented visual culture understanding\.ArXivabs/2505\.14462\.External Links:[Link](https://api.semanticscholar.org/CorpusID:278769446)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px1.p1.1)\.
- Liet al\.\(2025c\)K\. Li, Z\. Zhang, H\. Yin, L\. Zhang, L\. Ou, J\. Wu, W\. Yin, B\. Li, Z\. Tao, X\. Wang, W\. Shen, J\. Zhang, D\. Zhang, X\. Wu, Y\. Jiang, M\. Yan, P\. Xie, F\. Huang, and J\. ZhouWebSailor: navigating super\-human reasoning for web agent\.ArXivabs/2507\.02592\.External Links:[Link](https://api.semanticscholar.org/CorpusID:280078605)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px2.p1.1)\.
- Liet al\.\(2025d\)S\. Li, X\. Bu, W\. Wang, J\. Liu, J\. Dong, H\. He, H\. Lu, H\. Zhang, C\. Jing, Z\. Li, C\. Li, J\. Tian, C\. Zhang, T\. Peng, Y\. He, J\. Gu, Y\. Zhang, J\. Yang, G\. Zhang, W\. Huang, W\. Zhou, Z\. Zhang, R\. Ding, and S\. WenMM\-browsecomp: a comprehensive benchmark for multimodal browsing agents\.External Links:2508\.13186,[Link](https://arxiv.org/abs/2508.13186)Cited by:[§F\.6](https://arxiv.org/html/2609.11318#A6.SS6.p2.1),[Figure 1](https://arxiv.org/html/2609.11318#S1.F1),[Figure 1](https://arxiv.org/html/2609.11318#S1.F1.4),[§1](https://arxiv.org/html/2609.11318#S1.p2.1),[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2609.11318#S2.T1.7.1.3.1),[§4\.3\.4](https://arxiv.org/html/2609.11318#S4.SS3.SSS4.p1.1)\.
- Liet al\.\(2025e\)X\. Li, G\. Dong, J\. Jin, Y\. Zhang, Y\. Zhou, Y\. Zhu, P\. Zhang, and Z\. DouSearch\-o1: agentic search\-enhanced large reasoning models\.InConference on Empirical Methods in Natural Language Processing,External Links:[Link](https://api.semanticscholar.org/CorpusID:275405676)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px2.p1.1)\.
- Liuet al\.\(2024\)Y\. Liu, H\. Duan, Y\. Zhang, B\. Li, S\. Zhang, W\. Zhao, Y\. Yuan, J\. Wang, C\. He, Z\. Liu, K\. Chen, and D\. LinMMBench: is your multi\-modal model an all\-around player?\.InEuropean Conference on Computer Vision,External Links:[Link](https://arxiv.org/abs/2307.06281)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px1.p1.1)\.
- Luet al\.\(2025\)R\. Lu, Z\. Hou, Z\. Wang, H\. Zhang, X\. Liu, Y\. Li, S\. Feng, J\. Tang, and Y\. DongDeepDive: advancing deep search agents with knowledge graphs and multi\-turn rl\.External Links:2509\.10446,[Link](https://arxiv.org/abs/2509.10446)Cited by:[§A\.4](https://arxiv.org/html/2609.11318#A1.SS4.p1.1)\.
- Masryet al\.\(2022\)A\. Masry, D\. X\. Long, J\. Q\. Tan, S\. Joty, and E\. HoqueChartQA: a benchmark for question answering about charts with visual and logical reasoning\.InFindings of the Association for Computational Linguistics: ACL 2022,S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),Dublin, Ireland,pp\. 2263–2279\.External Links:[Link](https://aclanthology.org/2022.findings-acl.177/),[Document](https://dx.doi.org/10.18653/v1/2022.findings-acl.177)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px1.p1.1)\.
- Mathewet al\.\(2020\)M\. Mathew, D\. Karatzas, R\. Manmatha, and C\. V\. JawaharDocVQA: a dataset for vqa on document images\.2021 IEEE Winter Conference on Applications of Computer Vision \(WACV\),pp\. 2199–2208\.External Links:[Link](https://api.semanticscholar.org/CorpusID:220280200)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px1.p1.1)\.
- Mialonet al\.\(2024\)G\. Mialon, C\. Fourrier, T\. Wolf, Y\. LeCun, and T\. ScialomGAIA: a benchmark for general ai assistants\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=fibxvahvs3)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- Microsoft \(2024\)MicrosoftMicrosoft copilot\.External Links:[Link](https://copilot.microsoft.com/)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px2.p1.1)\.
- Nakanoet al\.\(2021\)R\. Nakano, J\. Hilton, S\. Balaji, J\. Wu, O\. Long, C\. Kim, C\. Hesse, S\. Jain, V\. Kosaraju, W\. Saunders, X\. Jiang, K\. Cobbe, T\. Eloundou, G\. Krueger, K\. Button, M\. Knight, B\. Chess, and J\. SchulmanWebGPT: browser\-assisted question\-answering with human feedback\.ArXivabs/2112\.09332\.External Links:[Link](https://api.semanticscholar.org/CorpusID:245329531)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px2.p1.1)\.
- Ninget al\.\(2026\)X\. Ning, D\. Fu, T\. Wei, M\. Ai, J\. Zou, T\. Li, H\. Tong, Y\. Zhu, H\. Hamann, and J\. HeMC\-search: evaluating and enhancing multimodal agentic search with structured long reasoning chains\.External Links:2603\.00873,[Link](https://arxiv.org/abs/2603.00873)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2609.11318#S2.T1.7.1.7.1)\.
- OpenAI \(2025\)OpenAIIntroducing deep research\.External Links:[Link](https://openai.com/index/introducing-deep-research/)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px2.p1.1)\.
- Perplexity AI \(2025\)Perplexity AIIntroducing perplexity deep research\.External Links:[Link](https://www.perplexity.ai/hub/blog/introducing-perplexity-deep-research)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px2.p1.1)\.
- Schicket al\.\(2023\)T\. Schick, J\. Dwivedi\-Yu, R\. Dessì, R\. Raileanu, M\. Lomeli, L\. Zettlemoyer, N\. Cancedda, and T\. ScialomToolformer: language models can teach themselves to use tools\.ArXivabs/2302\.04761\.External Links:[Link](https://api.semanticscholar.org/CorpusID:256697342)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px2.p1.1)\.
- Songet al\.\(2025\)H\. Song, J\. Jiang, Y\. Min, J\. Chen, Z\. Chen, W\. X\. Zhao, L\. Fang, and J\. WenR1\-searcher: incentivizing the search capability in llms via reinforcement learning\.ArXivabs/2503\.05592\.External Links:[Link](https://api.semanticscholar.org/CorpusID:276884818)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px2.p1.1)\.
- Taoet al\.\(2026\)X\. Tao, Y\. Teng, X\. Su, X\. Fu, J\. Wu, C\. Tao, Z\. Liu, H\. Bai, R\. Liu, and L\. KongMMSearch\-plus: benchmarking provenance\-aware search for multimodal browsing agents\.External Links:2508\.21475,[Link](https://arxiv.org/abs/2508.21475)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2609.11318#S2.T1.7.1.4.1)\.
- Teamet al\.\(2025\)V\. Team, W\. Hong, W\. Yu, X\. Gu, G\. Wang, G\. Gan, H\. Tang, J\. Cheng, J\. Qi, J\. Ji, L\. Pan, S\. Duan, W\. Wang, Y\. Wang, Y\. Cheng, Z\. He, Z\. Su, Z\. Yang, Z\. Pan, A\. Zeng, B\. Wang, B\. Chen, B\. Shi, C\. Pang, C\. Zhang, D\. Yin, F\. Yang, G\. Chen, J\. Xu, J\. Zhu, J\. Chen, J\. Chen, J\. Chen, J\. Lin, J\. Wang, J\. Chen, L\. Lei, L\. Gong, L\. Pan, M\. Liu, M\. Xu, M\. Zhang, Q\. Zheng, S\. Yang, S\. Zhong, S\. Huang, S\. Zhao, S\. Xue, S\. Tu, S\. Meng, T\. Zhang, T\. Luo, T\. Hao, T\. Tong, W\. Li, W\. Jia, X\. Liu, X\. Zhang, X\. Lyu, X\. Fan, X\. Huang, Y\. Wang, Y\. Xue, Y\. Wang, Y\. Wang, Y\. An, Y\. Du, Y\. Shi, Y\. Huang, Y\. Niu, Y\. Wang, Y\. Yue, Y\. Li, Y\. Zhang, Y\. Wang, Y\. Wang, Y\. Zhang, Z\. Xue, Z\. Hou, Z\. Du, Z\. Wang, P\. Zhang, D\. Liu, B\. Xu, J\. Li, M\. Huang, Y\. Dong, and J\. TangGLM\-4\.5v and glm\-4\.1v\-thinking: towards versatile multimodal reasoning with scalable reinforcement learning\.External Links:2507\.01006,[Link](https://arxiv.org/abs/2507.01006)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px1.p1.1)\.
- Weiet al\.\(2025\)J\. Wei, Z\. Sun, S\. Papay, S\. McKinney, J\. Han, I\. Fulford, H\. W\. Chung, A\. T\. Passos, W\. Fedus, and A\. GlaeseBrowseComp: a simple yet challenging benchmark for browsing agents\.External Links:2504\.12516,[Link](https://arxiv.org/abs/2504.12516)Cited by:[§1](https://arxiv.org/html/2609.11318#S1.p2.1),[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2609.11318#S2.T1.7.1.2.1)\.
- Wonget al\.\(2025\)R\. Wong, J\. Wang, J\. Zhao, L\. Chen, Y\. Gao, L\. Zhang, X\. Zhou, Z\. Wang, K\. Xiang, G\. Zhang, W\. Huang, Y\. Wang, and K\. WangWideSearch: benchmarking agentic broad info\-seeking\.External Links:2508\.07999,[Link](https://arxiv.org/abs/2508.07999)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- xAI \(2025\)xAIGrok 3 beta: the age of reasoning agents\.External Links:[Link](https://x.ai/news/grok-3)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px2.p1.1)\.
- Xiet al\.\(2025\)Y\. Xi, J\. Lin, M\. Zhu, Y\. Xiao, Z\. Ou, J\. Liu, T\. Wan, B\. Chen, W\. Liu, Y\. Wang, R\. Tang, W\. Zhang, and Y\. YuInfoDeepSeek: benchmarking agentic information seeking for retrieval\-augmented generation\.External Links:2505\.15872,[Link](https://arxiv.org/abs/2505.15872)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- Xieet al\.\(2024\)T\. Xie, D\. Zhang, J\. Chen, X\. Li, S\. Zhao, R\. Cao, T\. J\. Hua, Z\. Cheng, D\. Shin, F\. Lei, Y\. Liu, Y\. Xu, S\. Zhou, S\. Savarese, C\. Xiong, V\. Z\. Zhou, T\. Wu, and C\. L\. HuOSWorld: benchmarking multimodal agents for open\-ended tasks in real computer environments\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://arxiv.org/abs/2404.07972)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- Xuet al\.\(2025\)Y\. Xu, X\. Long, Z\. Zheng, and J\. GaoRAVine: reality\-aligned evaluation for agentic search\.External Links:2507\.16725,[Link](https://arxiv.org/abs/2507.16725)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- Yaoet al\.\(2022\)S\. Yao, H\. Chen, J\. Yang, and K\. NarasimhanWebShop: towards scalable real\-world web interaction with grounded language agents\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://arxiv.org/abs/2207.01206)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- Yaoet al\.\(2023\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=WE_vluYUL-X)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px2.p1.1)\.
- Yeet al\.\(2026\)F\. Ye, Y\. Hu, P\. Zhu, Y\. Li, Z\. Jin, Y\. Xiao, Y\. Wang, L\. Wang, Z\. Zhang, L\. Wang, Y\. Deng, B\. Wang, Y\. Zhang, L\. Su, X\. Wang, H\. Zhao, C\. Wei, Q\. Ren, B\. Hooi, A\. Bo, S\. Yan, and L\. BingMiroEval: benchmarking multimodal deep research agents in process and outcome\.External Links:2603\.28407,[Link](https://arxiv.org/abs/2603.28407)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- Yueet al\.\(2023\)X\. Yue, Y\. Ni, K\. Zhang, T\. Zheng, R\. Liu, G\. Zhang, S\. Stevens, D\. Jiang, W\. Ren, Y\. Sun, C\. Wei, B\. Yu, R\. Yuan, R\. Sun, M\. Yin, B\. Zheng, Z\. Yang, Y\. Liu, W\. Huang, H\. Sun, Y\. Su, and W\. ChenMMMU: a massive multi\-discipline multimodal understanding and reasoning benchmark for expert agi\.2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 9556–9567\.External Links:[Link](https://api.semanticscholar.org/CorpusID:265466525)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2026\)H\. Zhang, J\. Zhou, B\. Li, B\. Zhou, Y\. Shan, H\. Lu, Z\. Cao, J\. Chen, Y\. Han, Z\. Sheng, Z\. Tao, H\. Liang, J\. Wu, Y\. Shi, Y\. He, J\. Lin, Q\. Zhang, G\. Yan, R\. Zhao, Z\. Li, X\. Yu, L\. Mei, C\. Chen, W\. Zhang, and B\. CuiBrowseComp\-V3V^\{3\}: a visual, vertical, and verifiable benchmark for multimodal browsing agents\.External Links:2602\.12876,[Link](https://arxiv.org/abs/2602.12876)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2609.11318#S2.T1.7.1.5.1)\.
- Zhanget al\.\(2025\)W\. Zhang, Y\. Li, Y\. Bei, J\. Luo, G\. Wan, L\. Yang, C\. Xie, Y\. Yang, W\. Huang, C\. Miao, H\. P\. Zou, X\. Luo, Y\. Zhao, Y\. Chen, C\. Chan, P\. Zhou, X\. Zhang, C\. Zhang, J\. Shang, M\. Zhang, Y\. Song, I\. King, and P\. S\. YuFrom web search towards agentic deep research: incentivizing search with reasoning agents\.ArXivabs/2506\.18959\.External Links:[Link](https://api.semanticscholar.org/CorpusID:279999700)Cited by:[§1](https://arxiv.org/html/2609.11318#S1.p1.1),[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px2.p1.1)\.
- Zhenget al\.\(2025\)Y\. Zheng, D\. Fu, X\. Hu, X\. Cai, L\. Ye, P\. Lu, and P\. LiuDeepResearcher: scaling deep research via reinforcement learning in real\-world environments\.InConference on Empirical Methods in Natural Language Processing,External Links:[Link](https://api.semanticscholar.org/CorpusID:277596185)Cited by:[§1](https://arxiv.org/html/2609.11318#S1.p1.1),[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px2.p1.1)\.
- Zhouet al\.\(2025\)P\. Zhou, B\. Leon, X\. Ying, C\. Zhang, Y\. Shao, Q\. Ye, D\. Chong, Z\. Jin, C\. Xie, M\. Cao,et al\.BrowseComp\-zh: benchmarking web browsing ability of large language models in chinese\.External Links:2504\.19314,[Link](https://arxiv.org/abs/2504.19314)Cited by:[§1](https://arxiv.org/html/2609.11318#S1.p2.1),[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.
- Zhouet al\.\(2024\)S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried, U\. Alon, and G\. NeubigWebArena: a realistic web environment for building autonomous agents\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=oKn9c6ytLx)Cited by:[§2](https://arxiv.org/html/2609.11318#S2.SS0.SSS0.Px3.p1.1)\.

## Appendix AConstruction Workflow and Internal Interfaces

This appendix provides supporting details on implementation and analysis, including the annotation and evaluation interfaces, the AI\-assisted construction procedure, metric and judge specifications, model configurations, robustness checks, additional benchmark statistics, diagnostic analyses, case studies, and release considerations\.

### A\.1Annotation and Quality\-Control Interface

The internal annotation platform supports collaborative data construction and quality control\. Within a single interface, annotators create and revise benchmark items by editing the question, short answer, hidden Node\-Relation design, checklist, sources, images, category, authored\-template label, and operation tags\. The platform also supports Markdown rendering of the design field, image inspection, secondary\-audit comments and ratings, and benchmark locking\. Together, these functions operationalize the validation protocol: an independent auditor can check whether the question leaks the path, whether each checklist item constitutes a verifiable conclusion, whether each source supports a necessary node or relation, and whether uploaded images or other non\-text evidence are used in the chain\.

![Refer to caption](https://arxiv.org/html/2609.11318v1/figures/webpage/webpage-add-new.png)Figure 8:Annotation interface for collaborative Mr\.LHDR construction\. Before an item passes the secondary audit and is locked, the system allows annotators to inspect and edit its question, answer, design, checklist, sources, images, and metadata\.
### A\.2Evaluation Interface

The evaluation platform supports launching model runs, monitoring their status, inspecting model responses, and auditing judge outputs\. For each item–model pair, the platform records whether the run completed or failed, the model response, the judge’s final\-answer decision, and the checklist completion vector\.

![Refer to caption](https://arxiv.org/html/2609.11318v1/figures/webpage/webpage-eval.png)Figure 9:Evaluation interface for launching runs and inspecting their status, responses, judge decisions, checklist completion, and aggregate metrics\. The platform serves as an execution and audit tool\.
### A\.3Dataset Schema and Construction Procedure

Table 5:Core fields and annotations in each Mr\.LHDR item\. Thechecklistrecords necessary intermediate conclusions rather than browser actions; the checklist DAG records their verified prerequisite relations;sourcessupport the nodes, relations, checklist items, and final answer;imagesprovide non\-textual entry points or key evidence\.A node consists of a core entity paired with searchable and verifiable properties, such as a photo, map location, paper PDF, official document, logo, chart, timestamp, work, affiliation, venue, or founder\. A relation uses a verified property of one node as the entry condition for another node\.

### A\.4AI\-Assisted Construction Pipeline

AI assistance follows an expand\-then\-constrain process: the model broadens the candidate design space, while annotators determine which elements remain as benchmark evidence\. The first layer consists of human\-controlled task specification: before any model generation, annotators define the target category, operation tags, and golden\-example style\. Within these constraints, AI supports graph\-level exploration by proposing Node\-Relation graphs comprising five to six information\-rich nodes, verifiable properties, possible multimodal sources, and bridge relations\. AI is then used for checklist\-level stress testing, expanding the graph into an intentionally over\-complete set of candidate conclusions, often spanning 25–28 steps, to expose missing evidence, weak dependencies, duplicate facts, and potential shortcuts\. Following DeepDive’s*Blur Attribute\-Rich Path*strategy\([Lu et al\., 2025](https://arxiv.org/html/2609.11318#bib.bib21)\), annotators also use AI to propose coarser\-grained renderings of high\-information attributes while preserving the unique convergence of the reasoning path: an exact year such as 1978 may be rewritten as “the late 1970s,” while overly direct place or affiliation cues may be abstracted into recoverable descriptors\. Any blur is rejected if it introduces another valid answer or renders a necessary bridge unverifiable\. Annotators complete this layer by compressing the candidate chain: they remove background facts, merge redundant conclusions, and retain only the intermediate conclusions necessary to derive the final answer\.

The second layer converts the pruned chain into a grounded benchmark item\. Annotators draft or revise the natural\-language question, short answer, source list, image evidence, and metadata while verifying that inspectable evidence supports every necessary conclusion\. Question polishing is treated as leakage control rather than stylistic rewriting: annotators remove intermediate\-node hints, direct\-answer cues, and wording that reveals the intended search path\. The internal annotation platform facilitates collaborative editing, secondary\-audit ratings and comments, image inspection, and the benchmark admission workflow\. Thus, AI provides breadth during candidate exploration, whereas human annotators determine factual support, checklist irreducibility, multimodal necessity, shortcut resistance, label correctness, and final admission to the evaluation set\.

The pipeline is interactive and does not correspond to a single generation prompt\. Model assistance is introduced at several distinct points—task specification, graph\-level exploration, checklist stress testing, and blur rewriting\. Each point is conditioned on the item under construction and guided by an annotator who accepts, edits, or discards the output\. The expansion stage deliberately over\-produces candidate checklists of 25–28 steps; the 102 admitted items retain 1,231 checklist steps in total, averaging 12\.1 per item\. Fewer than half of the proposed conclusions survive pruning, consistent with the expand\-then\-constrain design, in which annotators remove most model proposals\.

## Appendix BBenchmark Statistics and Composition

Table 6:Overview statistics for the Mr\.LHDR benchmark\. Operation tags are multi\-label, so their percentages need not sum to 100%\. Non\-text evidence coverage counts either uploaded image assets or source URLs pointing to PDF, video, or map evidence\.Table 7:Benchmark split statistics\. Operation tags are multi\-label, so tag counts and percentages do not sum to 102 items or 100%\.ScopeGroupItemsPercentAvg\. checklistAvg\. sourcescategoryacademia109\.812\.38\.7categorygeography1716\.714\.49\.2categorymedia2120\.611\.98\.7categoryorganizations1312\.711\.27\.2categorypeople1211\.811\.06\.6categorysociety109\.811\.26\.7categorysports98\.812\.18\.0categorytechnology109\.811\.68\.1authored templateLinear7371\.611\.87\.9authored templateMulti\-branch2928\.412\.88\.3operation tagConstraint7674\.511\.98\.1operation tagNumerical3029\.411\.87\.0operation tagSymbolic8179\.412\.28\.2operation tagTemporal1817\.611\.67\.2checklist length103534\.310\.07\.3checklist length11–123433\.311\.47\.6checklist length13–152322\.513\.79\.1checklist length16–1865\.916\.78\.8checklist length19\+43\.920\.010\.8dependency depth6–93534\.311\.18\.1dependency depth10–113837\.311\.17\.7dependency depth12–182928\.414\.48\.3source count1–62726\.511\.65\.0source count7–83736\.311\.37\.6source count9–102524\.512\.49\.6source count11\+1312\.714\.712\.5Table[6](https://arxiv.org/html/2609.11318#A2.T6)summarizes the Mr\.LHDR evaluation set, while Table[7](https://arxiv.org/html/2609.11318#A2.T7)presents benchmark diagnostics at the split level\. The benchmark currently contains 102 items\. Their checklists comprise 1,231 necessary intermediate conclusions in total, with a mean length of 12\.1, a median length of 11, and a range of 10 to 21 steps\. In the author\-verified checklist DAGs, dependency depth—defined as the maximum number of conclusions along a prerequisite path, including both endpoints—has a mean of 10\.4, a median of 10, and a range of 6 to 18\. The evidence fields are similarly dense: the benchmark includes 818 source URLs, with an average of 8\.0 URLs per item, as well as 101 image assets\. Together, the short answers, complete checklist annotations, explicit dependencies, and evidence fields make Mr\.LHDR verifiable at both the final\-answer and process levels\.

The construction metadata covers all eight target categories and both coarse authored\-template labels\. The largest category is media, with 21 items, followed by geography with 17; the smallest is sports, with 9 items\. Each remaining category contains between 10 and 13 items\. Most items follow the Linear template: 73 items \(71\.6%\) use this template, while 29 items \(28\.4%\) use the Multi\-branch template, which was designed to emphasize multiple evidence paths and their subsequent joint use\. Operation tags characterize local demands within a chain rather than mutually exclusive task families: Symbolic appears in 81 items \(79\.4%\), Constraint in 76 \(74\.5%\), Numerical in 30 \(29\.4%\), and Temporal in 18 \(17\.6%\)\. Because these tags are multi\-label, their percentages need not sum to 100%\.

Dependency depth and checklist length capture complementary structural properties, rather than the length of browsing traces or the verbosity of prompts\. Dependency depth measures the longest sequential prerequisite path, whereas checklist length measures the total number of necessary conclusions, including those on parallel branches\. By construction, no benchmark item has fewer than 10 checklist steps\. Most items fall into the 10–12\-step bucket \(69 items, 67\.6%\); 23 items \(22\.5%\) contain 13–15 steps, and 10 items \(9\.8%\) contain 16 or more steps\. Longer checklists also tend to have denser source evidence: items with 16 or more checklist steps use an average of 9\.6 source URLs, compared with 7\.4 URLs for items in the 10–12\-step bucket\.

For subsequent analysis, Mr\.LHDR tracks category, authored\-template label, operation tags, checklist length, source count, image availability, coarse\-grained non\-text evidence coverage, and Wikipedia usage\. Every item contains non\-text evidence when uploaded images are considered together with PDF, video, or map source URLs\. PDF, video, or map URLs appear in 45 items \(44\.1%\), while Wikipedia accounts for 241 of the 818 source URLs \(29\.5%\)\. These statistics support the claim that Mr\.LHDR cannot be reduced to a Wikipedia\-only or text\-only browsing benchmark\.

## Appendix CEvaluation Configuration and Failure Handling

### C\.1Model and Provider Configuration

Table[8](https://arxiv.org/html/2609.11318#A3.T8)lists the complete model configuration used in the evaluation\. The evaluation was conducted from April 30 to May 5, 2026 \(UTC\)\. Regular OpenRouter models in the search\+image group were evaluated through the benchmark interface with image evidence and, when exposed by the provider route, web\-search capability; image\-only models received image input but no search tool\. Deep research systems were evaluated through their dedicated research endpoints over the same OpenRouter route, whereas framework agents like[ByteDance \(2026\)](https://arxiv.org/html/2609.11318#bib.bib15)were evaluated through DeerFlow backends\. We implemented neither a ReAct loop nor any harness\-level optimizations\. Each run consisted of a single request containing the question verbatim\. For the tool\-augmented group, the request additionally declared the provider’s native server\-side search tool, ensuring that all browsing was executed and completed within the provider’s service before the response was returned to us\. We provided no system prompt and set no sampling parameters\. Because the providers’ own routes were used, the deep\-research endpoints did not accept inline image parts; these systems therefore received the image evidence as a URL in the prompt text rather than as attached bytes\.

Table 8:Full model configuration\. All reported metrics use the same fixed 102\-item denominator\.

## Appendix DMetrics and Judge Specification

### D\.1Interpretation of CS and DACS

Following prior checklist\-based evaluations in browsing benchmarks, CS provides a useful measure of broad coverage but is not well aligned with long\-horizon deep research because it treats checklist items as unordered and interchangeable\. In Mr\.LHDR, downstream conclusions are not independent local facts: they are warranted only after the preceding entities, constraints, and disambiguating relations have been established\. When these prerequisites are absent, downstream statements that match individual checklist items should not be interpreted as evidence of successful reasoning, because they may reflect unsupported guesses, search shortcuts, or chance agreement rather than a valid continuation of the evidence path\. CS can therefore overestimate process reliability by aggregating local matches across a broken dependency graph\. DACS addresses this limitation while retaining credit for genuinely independent branches\.

The DACS recursion defined in Section[4\.1\.3](https://arxiv.org/html/2609.11318#S4.SS1.SSS3)scores each verified dependency DAG directly, independently of its coarse authored\-template label\. Independent subgraphs retain credit separately, whereas a merge node receives credit only when the node itself and all required branch dependencies are satisfied\. DACS measures whether the submitted response states dependency\-consistent conclusions\.

### D\.2Automatic Judge Specification

The automatic judge receives the user question, reference answer, reference checklist, and model response\. The metric script reads only two machine\-readable fields from the judge output:OVERALL\_CORRECTNESSandCHECKLIST\_RESULT\. The judge model in the current pipeline is Qwen3\-VL\-235B \(Qwen/Qwen3\-VL\-235B\-A22B\-Instruct\-FP8\)\. When the core answer is unchanged, answer normalization permits harmless differences in capitalization, punctuation, spelling variants, aliases, units, and brief paraphrases\. A checklist item receives credit only when the response states the corresponding conclusion completely and associates it with the correct entity and constraint; vague mentions, incorrect entity associations, missing qualifiers, and descriptions of search actions do not count as completion\. All reported verdicts are model\-produced; neither manual result auditing nor calibration against human labels is performed\.

### D\.3Exact LLM Judge Prompt

Figure[10](https://arxiv.org/html/2609.11318#A4.F10)presents the exact prompt used by the automatic judge, including the required machine\-readable fields for final\-answer correctness and checklist completion\.

You are an AI evaluator\. Your task is to evaluate the quality of an answer\. I will provide you with the user’s question, the reference answer \(ground truth\), a checklist, and the answer to be evaluated\.\-\-\- USER QUESTION \-\-\- \{question\}\-\-\- REFERENCE ANSWER \(Ground Truth\) \-\-\- \{reference\_answer\} This reference answer is considered the correct and ideal response content\-wise\.\-\-\- REFERENCE CHECKLIST \-\-\- \{checklist\_lines\}\-\-\- MODEL’S GENERATED ANSWER TO EVALUATE \-\-\- \{generated\_answer\}\-\-\- EVALUATION INSTRUCTIONS \-\-\- Please provide your evaluation strictly in the following format on separate lines: 1\. Checklist Score: Determine how many of the \{n\} items in the REFERENCE CHECKLIST have been correctly and completely addressed\. The model’s generated answer must fully comply with an item for it to be considered complete\. State as ’CHECKLIST\_SCORE: \[correct\_items\]/\{n\}’ \(e\.g\., CHECKLIST\_SCORE: \{example\_score\}\)\. 2\. Checklist Result Vector: Provide a 0\-1 vector indicating whether each checklist item passed, in order\. ’1’ means fully satisfied, ’0’ means not fully satisfied\. Output as ’CHECKLIST\_RESULT: \[1,0,1\]’\. 3\. Overall Correctness: Judge whether the generated answer is consistent with the reference answer in core content\. Content consistency is key; minor wording differences are acceptable\. State as ’OVERALL\_CORRECTNESS: \[YES/NO\]’\.Provide only these three formatted lines as your response\.

Figure 10:LLM judge prompt used to evaluate final\-answer correctness and checklist completion\. The judge outputs only the formatted fields shown above; OA, SA, CS, and DACS are computed deterministically from these outputs and the frozen dependency graphs\.

## Appendix EStatistical and Cross\-Judge Robustness

### E\.1Item\-Bootstrap Uncertainty

Given an evaluation set of 102 items, Table[9](https://arxiv.org/html/2609.11318#A5.T9)complements the per\-model point estimates with 95% confidence intervals obtained by resampling items \(B=10000B=10000, seed 20260731\)\. For GPT\-5\.5, the system with the highest OA, the estimate of 43\.1% has an interval of \[33\.3, 52\.9\], spanning twenty percentage points\. The OA ordering of GPT\-5\.5 and o4\-mini, which have the two highest point estimates, is preserved in 96\.4% of resamples, exceeding 95%\. Among GPT\-5\.4, o3, and Gemini 3\.1 Pro, the maximum pairwise separability is 67\.1%; by comparison, a difference that never reverses would yield 100%\. These results provide the strongest evidence of separation for the leading pair, while indicating that the remaining per\-model rankings should be interpreted with appropriate caution\.

Table 9:Full Mr\.LHDR results with 95% bootstrap confidence intervals over the 25 systems\. Point estimates match Table[4](https://arxiv.org/html/2609.11318#S4.T4); intervals are the 2\.5/97\.5 percentiles ofB=10000B=10000bootstrap resamples of the 102 items \(seed 20260731\)\. Wide intervals and heavy overlap indicate that per\-model rankings should not be over\-interpreted; Figure[11](https://arxiv.org/html/2609.11318#A5.F11)gives paired uncertainty for the within\-system metric penalties\.Figure[11](https://arxiv.org/html/2609.11318#A5.F11)reports paired uncertainty for the answer\-completeness penalty OA−\-SA: the proportion of all items for which the final answer is correct but the response still omits at least one required conclusion\. Because SA is nested within OA, this difference is non\-negative by construction; the intervals therefore quantify uncertainty in the*magnitude*of the penalty, not evidence about its direction\. When expressed only among items with correct final answers, the penalty reaches 72\.7%: for the most affected system, more than eight in ten correct answers are accompanied by responses that omit at least one required conclusion\. The figure’s rightmost column also reports the dependency penalty CS−\-DACS, namely the checklist credit removed when prerequisite conclusions are absent\. We present this penalty as text rather than in a second plotted panel because its point estimates span only 3\.0 to 8\.9 percentage points and every interval overlaps every other interval; a second panel would therefore show no separable differences\. Moreover, the two penalties use different units of evaluation—whole\-item correctness and average checklist credit—so their magnitudes should not be interpreted as a direct statistical comparison\.

Figure 11:Paired item\-bootstrap intervals for the answer\-completeness penalty OA−\-SA among systems with OA≥\\geq5%, ordered by the penalty itself\. Each marker denotes a point estimate, and each line denotes the 95% percentile interval across 10000 resamples\. Marker area is proportional to the system’s OA, which is printed after the model name and serves as the denominator when the penalty is expressed as a conditional share\. The third column presents this conditional share among items with correct final answers\. Because the gap is non\-negative by construction, an interval reaching zero means that the penalty may be absent, not that its sign is uncertain\. For the same systems, the rightmost column reports the dependency penalty CS−\-DACS \(point estimate and 95% CI\) as text rather than as a plotted panel because its intervals overlap too heavily to reveal separable differences\.
### E\.2Complete Judge Sampling Frame and Agreement Analysis

Because SA and DACS depend on checklist\-level judgments, every process\-level claim in this paper depends on the judge\. Reporting results from only one judge would leave this dependence untested\. We therefore re\-score cached responses from a subset of the evaluated systems \(2065 responses, 24782 checklist steps\) using 12 judges drawn from 6 model families\. The scoring prompt and parsing procedure are held fixed across judges; only the model issuing the verdict varies\.

Because vendors contribute unequal numbers of systems, we report agreement on a family\-balanced subsample\. The stratified grid selects the most recent flagship from each model family as its prespecified representative and covers all 102 items across all 8 categories\. This yields 702 responses and 8417 checklist steps scored by every judge \(Table[10](https://arxiv.org/html/2609.11318#A5.T10)\)\.

Agreement is high for both same\-vendor and cross\-vendor judge pairs\. Pairwise final\-answer agreement ranges from 95\.9% to 99\.7%, with Cohen’sκ\\kapparanging from 0\.845 to 0\.989\. The weakest pair is cross\-vendor: among the 58 cross\-vendor pairs, the minimum agreement is 95\.9% \(κ=0\.845\\kappa=0\.845\)\. Fleiss’κ\\kappais 0\.940 for final answers \(95% CI \[0\.918, 0\.958\] over 4000 cell\-level bootstrap resamples\) and 0\.803 for individual checklist steps; all judges are unanimous on 94\.3% of responses\. A sensitivity analysis that replaces each family’s flagship with its*weakest*member yieldsκ=0\.909\\kappa=0\.909\[0\.877, 0\.937\]\. We also pool 10 judges into a single reference verdict for 2157 responses, counting a response as correct when at least 50% judges in the pool agree\. The pool is unanimous on 97\.5% of responses\. The reported judge—which is not a member of the pool—matches this reference on 97\.1% of final answers \(κ=0\.890\\kappa=0\.890\) and 86\.6% of checklist steps \(κ=0\.728\\kappa=0\.728\), as shown in Table[11](https://arxiv.org/html/2609.11318#A5.T11)\.

#### E\.2\.1Detailed Agreement Results

Table[10](https://arxiv.org/html/2609.11318#A5.T10)reports all pairwise agreement results underlying this analysis: raw final\-answer agreement appears below the diagonal and Cohen’sκ\\kappaabove it for all\(122\)\\binom\{12\}\{2\}judge pairs on the family\-stratified grid\. The judge pool re\-scored cached responses from a subset of the evaluated systems \(2065 responses, 24782 checklist steps\), and these verdicts support the cross\-judge checks reported elsewhere in the paper\. The matrix is restricted to the stratified grid so that the agreement statistic does not depend on the number of systems contributed by each vendor\.

Table 10:Inter\-judge agreement on a family\-stratified model subset: one representative per model family, selected as the family’s most recent flagship \(OpenAI \(gpt\-5\.5\), Anthropic \(claude\-opus\-4\.7\), Google \(gemini\-3\.1\-pro\-preview\), Qwen \(qwen3\-vl\-32b\-instruct\), Meta \(llama\-4\-maverick\), Perplexity \(sonar\-deep\-research\), xAI \(grok\-4\.20\), Alibaba \(tongyi\-deepresearch\-30b\-a3b\)\), covering all 8 model families and all 102 items \(all 8 categories\) in the benchmark\. Of 799 candidate \(item, representative\) cells with a real, non\-empty response, 97 lack a verdict from at least one judge and are excluded, leaving a 702\-cell grid evaluated against 12 judges \(the reported judge, one additional Qwen judge, and the 10\-member cross\-vendor jury pool\)\. Columns are numbered as the rows; lower triangle: raw agreement on final answer \(OA\); upper triangle: Cohen’sκ\\kappa\. Cell shading encodes magnitude \(bluefor OA over a fixed 95–100% range;orangeforκ\\kappaover a fixed 0\.80–1\.00 range; darker = higher agreement\); exact values stay printed in every cell\. A representative\-selection sensitivity check \(each family’s weakest member instead of its flagship\) and bootstrap 95% CIs are reported in the accompanying macros\.Table[11](https://arxiv.org/html/2609.11318#A5.T11)compares each individual judge against the pooled LLM reference verdict\.

Table 11:Each judge against the pooled verdict of 10 models scoring the same responses with the identical prompt\. A response counts as correct for the pool only when at least 50% of the judges pass it\. Judges inside the pool agree with it by construction; the informative rows are the two that are not\.
#### E\.2\.2Judge Overlap with an Evaluated System

DeerFlow \(qwen3\-vl\-235b\) in Table[4](https://arxiv.org/html/2609.11318#S4.T4)uses the same base model as the judge that issues the reported verdicts, and Section[4\.3\.1](https://arxiv.org/html/2609.11318#S4.SS3.SSS1)evaluates this model directly\. Three measurements characterize the resulting risk of self\-evaluation\. First, DeerFlow \(qwen3\-vl\-235b\) achieves 15\.7% OA and 9\.8% SA, both among the lowest values in the table\. Second, in the ablation, this judge reports a DACS gain of 12\.6 points, below the 15\.0\-point median across the 10 judges from other vendors\. Third, the judge is excluded from the pooled reference used for comparison\. Within the full judge pool, the reported judge is the most lenient of the 12: it marks 16\.5% of final answers as correct and passes 50\.9% of checklist steps, whereas the next\-highest rates are 16\.1% and 50\.0%, respectively\. This leniency also explains the judge’s relative isolation in Table[10](https://arxiv.org/html/2609.11318#A5.T10): every pair involving it hasκ≤0\.870\\kappa\\leq 0\.870, below the minimum of0\.8770\.877among pairs that exclude it\. Consequently, the reported absolute process scores lie at the lenient end of the pool’s range, whereas the comparative conclusions rely on directions that are stable across judges\.

### E\.3Image Ablation: Cross\-Judge Self\-Evaluation Check

Because the ablation judge and the evaluated model share the same base model, we examine whether self\-preference affects the measured image gain\. We re\-score every frozen ablation response using all 12 judges from 6 vendor families and independently recompute the gain under each judge \(Table[12](https://arxiv.org/html/2609.11318#A5.T12)\)\. OA, CS, and DACS increase under every judge in the pool \(12, 12, and 12 out of 12, respectively\)\. For DACS, the self\-judge reports a 12\.6\-point gain, below the 15\.0\-point median across the 10 judges from other vendors\. The reported DACS effect therefore lies at the low end of the cross\-judge range\. SA is less stable: only 7 of the 12 judges report a positive gain\. This is consistent with the small number of items successful under this metric; accordingly, the conclusion about multimodality is based on OA, CS, and DACS\.

Table 12:Robustness of the image−\-text gain to the choice of judge\. The ablation responses are frozen; each row re\-scores all of them with a different judge and recomputes the gain\. The pool spans 6 vendor families and includes the subject model itself \(†\\dagger\), whose verdicts produced the numbers in Figure[5](https://arxiv.org/html/2609.11318#S4.F5)\. OA, CS and DACS gains are positive under every judge\. The self\-judge reports a*smaller*DACS gain than the cross\-family median, placing the reported effect at the low end of the cross\-judge range\. SA is less stable because its base rate is a handful of items and single\-verdict changes can flip its sign; the multimodality conclusion is therefore based on OA, CS, and DACS\.

## Appendix FSplit\-Level Performance and Failure Diagnostics

This appendix reports complete split\-level results by category, coarse authored\-template label \(Linear vs\. Multi\-branch\), operation tag \(Constraint, Symbolic, Numerical, and Temporal\), checklist length, and source count\. It also provides the protocol and full counts for the failure\-mode analysis\. Because operation tags are multi\-label, the corresponding splits should not be interpreted as disjoint partitions\.

### F\.1Performance by Category

Table[13](https://arxiv.org/html/2609.11318#A6.T13)reports complete results for all evaluated systems across the eight benchmark categories\.

Table 13:Model results by category\. Metrics are percentages over the items in each split; failed, missing, and invalid responses count as zero\.

### F\.2Performance by Authored Template

Figure 12:Performance of representative models by coarse authored template\. The bars compare OA and DACS for Linear\- and Multi\-branch\-template items\.We first partition Mr\.LHDR items by their coarse authored\-template labels\. The evaluation set contains 73 Linear\-template items, which account for 71\.6% of the benchmark, and 29 Multi\-branch\-template items, which account for 28\.4%\. On average, Multi\-branch\-template items have slightly longer checklists: 12\.8 steps, compared with 11\.8 steps for Linear\-template items\. Although this difference is consistent with the Multi\-branch template’s construction goal of combining multiple evidence routes, the label does not strictly determine the topology of an item’s verified DAG\.

As shown in Figure[12](https://arxiv.org/html/2609.11318#A6.F12), the authored\-template split does not produce a uniform OA gap: GPT\-5\.5 is lower on Multi\-branch\-template items, whereas Gemini 3 Flash and o3 Deep Research are slightly higher\. We interpret the split as a diagnostic rather than a standalone difficulty ranking or an exact partition of DAG topologies\. Although the Linear\- and Multi\-branch\-template subsets differ in graph topology on average, they also differ in category mix, source availability, and the positions of multimodal bridge steps\. The relevant diagnostic signal is that the Multi\-branch template tends to reveal whether an agent can maintain multiple partial conclusions until they are jointly needed, whereas the Linear template more directly tests whether an agent can avoid early drift along a predominantly sequential route\.

Table[14](https://arxiv.org/html/2609.11318#A6.T14)extends this comparison to all evaluated systems\.

Table 14:Model results by coarse authored template\. Metrics are percentages over the items in each split; failed, missing, and invalid responses count as zero\.
### F\.3Performance by Operation Tag

Table[15](https://arxiv.org/html/2609.11318#A6.T15)reports complete results by operation tag\. Because the tags are multi\-label, these columns describe overlapping subsets rather than a partition of the evaluation set\.

Table 15:Model results by operation tag\. Operation tags are multi\-label, so split counts do not sum to 102 items\. Metrics are percentages over the tagged items; failed, missing, and invalid responses count as zero\.
### F\.4Performance by Checklist Length

The main checklist\-length analysis appears in Section[4\.3\.2](https://arxiv.org/html/2609.11318#S4.SS3.SSS2); Table[16](https://arxiv.org/html/2609.11318#A6.T16)reports the complete results, including the small tail bins\.

Table 16:Model results by checklist\-length bucket\. Metrics are percentages over the items in each split; failed, missing, and invalid responses count as zero\.The 16–18 and 19\+ checklist\-length buckets contain only 6 and 4 items, respectively\. A single item can therefore change these cells by tens of percentage points, and a system can obtain high DACS without solving any items\. Consequently, per\-split orderings cannot establish that the longest checklist ranges are easier\. The difficulty trend reported in Section[4\.3\.2](https://arxiv.org/html/2609.11318#S4.SS3.SSS2)instead uses the better\-populated ranges, where individual items do not dominate the estimate\.

### F\.5Performance by Source Count

Table[17](https://arxiv.org/html/2609.11318#A6.T17)reports complete results by source\-count bucket\. Source count describes annotated evidence availability, not the number of browsing actions taken by a system or the amount of evidence it actually uses\.

Table 17:Model results by source\-count bucket\. Metrics are percentages over the items in each split; failed, missing, and invalid responses count as zero\.
### F\.6Failure\-Mode Analysis

This section presents the protocol, coverage, taxonomy, and full group\-level counts for the analysis summarized in Section[4\.3\.4](https://arxiv.org/html/2609.11318#S4.SS3.SSS4)\.

To characterize failures, we assign each unsupported checklist conclusion a diagnostic label from the MM\-BrowseComp taxonomy\([Li et al\., 2025d](https://arxiv.org/html/2609.11318#bib.bib20)\)\(Table[18](https://arxiv.org/html/2609.11318#A6.T18)\)\. To keep the failure definition consistent, checklist support for all 25 systems is determined using the frozen verdicts of Qwen3\.5\-Flash \(qwen3\.5\-flash\-2026\-02\-23\); this judge differs from the current judge used to report the performance metrics\. Qwen3\.6 Plus \(qwen3\.6\-plus\-2026\-04\-02\) then assigns a taxonomy label to each failed checklist statement\.

The analysis yields 30,775 checklist\-step decisions from 2,550 model–item runs, covering all 25 systems in Table[4](https://arxiv.org/html/2609.11318#S4.T4)\. Of these decisions, 17,409 steps are unsupported and receive failure labels; thus, 56\.6% of the required intermediate conclusions are unsupported, and 2,247 runs \(88\.1%\) contain at least one unsupported conclusion\. This rate is a step\-weighted micro\-average and is therefore not comparable with the per\-system CS values in the main table\.

Table 18:Taxonomy of failure modes used in our error analysis\.Table[19](https://arxiv.org/html/2609.11318#A6.T19)reports the complete failure\-mode counts and conditional distributions for each model group\.

Table 19:Distribution of failure modes assigned to failed checklist items, aggregated by model group\. Percent columns are conditioned on failed checklist items within each group; passed checklist items are excluded\. “Failed steps” counts the number of checklist entries with a non\-null label inevaluations\.checklist\_failure\_modes\.

## Appendix GCase Studies

The following audited case studies illustrate how final\-answer accuracy and checklist\-level process metrics differentiate among research behaviors\. They cover a complete solution to an item constructed with the Multi\-branch template, a late\-stage error in the reasoning chain that propagates to downstream evidence, and an early visual grounding failure followed by partial recovery through a chance guess\.

Table 20:Case Study Part 1/3: A multi\-branch success case\. The model correctly completes every checklist item and combines conclusions from the two branches to obtain the correct final answer\.FieldOriginal contentFigure![[Uncaptioned image]](https://arxiv.org/html/2609.11318v1/figures/case/6ae75518-1.png)QuestionThe classic cover design of this publisher, shown in the picture, is one of the most recognizable visual languages in British publishing history\. The publisher was founded by someone with family ties to an established publishing institution\. The name of this long\-standing institution originates from a historical figure who laid the foundation for a renowned university library in Britain\. This library features a famous tower displaying five classical orders of architecture\. Meanwhile, the publisher’s Penguin logo has been revised, and one of the founding partners of the design company responsible for the revision studied at an art school in London that bears a phoenix as its emblem\. How many years are there between the completion year of the tower and the graduation year of this designer?Modelopenai/gpt\-5\.5ResponseThe answer is337 years\.Reasoning chain:1\.The pictured publisher isPenguin Books; its 2003 Penguin logo refresh was done byPentagram Design\. \([https://www\.penguin\.co\.uk/discover/articles/penguin\-books\-logo\-history\-edward\-young\-allen\-lane](https://www.penguin.co.uk/discover/articles/penguin-books-logo-history-edward-young-allen-lane)\)2\.Penguin’s founder Allen Lane was tied toThe Bodley Head, which was named afterSir Thomas Bodley, founder of the Bodleian Library; Allen Lane succeeded John Lane, his uncle, at The Bodley Head before leaving to set up Penguin\. \([https://collections\.reading\.ac\.uk/special\-collections/collections/the\-bodley\-head\-ltd\-publisher/](https://collections.reading.ac.uk/special-collections/collections/the-bodley-head-ltd-publisher/)\)3\.The Bodleian’sTower of the Five Ordersreached its full five\-storey form and main outline byJune 1619, so I take the tower’s completion year as1619\. \([https://www\.oxoniensia\.org/volumes/1968/cole\.pdf](https://www.oxoniensia.org/volumes/1968/cole.pdf)\)4\.Pentagram was founded by five partners, includingAlan Fletcher\. \([https://www\.pentagram\.com/news/happy\-50th\-birthday\-pentagram](https://www.pentagram.com/news/happy-50th-birthday-pentagram)\)5\.Fletcher studied at theRoyal College of Artfrom1953 to 1956; the RCA’s phoenix is a recurring official emblem/tradition\. \([https://en\.wikipedia\.org/wiki/Alan\_Fletcher\_%28graphic\_designer%29](https://en.wikipedia.org/wiki/Alan_Fletcher_%28graphic_designer%29)\)Calculation:1956 \- 1619 = 337So, there are337 yearsbetween the tower’s completion year and Alan Fletcher’s graduation/end year\.Answer337Checklist✓The cover in the picture is recognized as Penguin Books’ classic orange three\-part cover design\.✓Penguin Books was founded by Allen Lane in 1935\.✓Allen Lane was the nephew of John Lane, who founded the publishing house The Bodley Head\.✓The name The Bodley Head derives from the logo featuring the head of Thomas Bodley\.✓Thomas Bodley was the founder of the Bodleian Library\.✓The Bodleian Library features a famous Tower of the Five Orders, displaying five classical architectural styles\.✓The Tower of the Five Orders was completed around 1619\.✓Penguin Books’ penguin logo was redesigned in 2003 by Angus Hyland, a partner at the design firm Pentagram\.✓One of Pentagram’s founding partners was Alan Fletcher\.✓Alan Fletcher studied at the Royal College of Art \(RCA\), whose emblem features a Phoenix\.✓Alan Fletcher graduated from RCA in 1956\.✓Calculating 1956 \- 1619 = 337 years\.Is\_Correct1Table 20:Case Study Part 1/3: A multi\-branch success case\. Continued\.Table 21:Case Study Part 2/3: A late\-chain propagation failure\. The model reasons correctly through checklist items 1–12, but an error at item 13 propagates through subsequent steps and leads to an incorrect final answer\.FieldOriginal contentFigure049365dc![[Uncaptioned image]](https://arxiv.org/html/2609.11318v1/figures/case/049365dc-1.png)QuestionThis elevated aqueduct bridge, spanning a mountain valley and closely associated with early British transportation engineering heritage, was designed with the involvement of an engineer; the same engineer is also credited with building a suspension bridge connecting the mainland and an island\. The waterway beneath the bridge belongs to a famous strait, and the island is known for its unique administrative status and diverse coastal landscapes\. On the island, there is a popular tourist spot with a well\-known place name frequently photographed by visitors, and its railway station sign also displays this long place name\. The coastal railway line at that station extends westward to a port town, where a ferry route crosses the Irish Sea to the capital port of Ireland\. The port reached by this route is located at the edge of a bay, with a long seawall extending into the sea at the port entrance; at the end of this seawall stands a lighthouse, a landmark that appeared in a YouTube video by a travel vlogger with the real name Paul aka\. The video features the full name of the club displayed on the wall of a house–what is that full name?Modelopenai/o4\-mini\-deep\-research\-2025\-06\-26ResponseThe clues all point to Anglesey \(linked by Telford’s Menai Suspension Bridge over the Menai Strait \([https://www\.britannica\.com/topic/Menai\-Bridge\#:~:text=Menai%20Bridge%2C%20suspension%20bridge%20,from%20masonry%20towers%20at%20either](https://www.britannica.com/topic/Menai-Bridge#:~:text=Menai%20Bridge%2C%20suspension%20bridge%20,from%20masonry%20towers%20at%20either)\)\)\. The long\-named village is Llanfairpwllgwyngyllgogerychwyrndrobwllllantysiliogogogoch \(the railway station sign famously bears the full name\) which lies on the line to Holyhead\. Holyhead’s huge breakwater \(2\.75 km long\) with its lighthouse at the end is well known \([https://lighthouses\.wales/the\-lighthouses/holyhead\-breakwater/\#:~:text=This%20lighthouse%20is%20perched%20at,by%20the%20Holyhead%20Port%20Authority](https://lighthouses.wales/the-lighthouses/holyhead-breakwater/#:~:text=This%20lighthouse%20is%20perched%20at,by%20the%20Holyhead%20Port%20Authority)\)\. In the travel video the wall of a house shows the letters “RAOB”–this stands for the club’s full name, theRoyal Antediluvian Order of Buffaloes\([https://www\.wikiwand\.com/en/articles/Royal\_Antediluvian\_Order\_of\_Buffaloes\#:~:text=The%20Royal%20Antediluvian%20Order%20of,to%20members](https://www.wikiwand.com/en/articles/Royal_Antediluvian_Order_of_Buffaloes#:~:text=The%20Royal%20Antediluvian%20Order%20of,to%20members)\)\.Answer:Royal Antediluvian Order of Buffaloes \([https://www\.wikiwand\.com/en/articles/Royal\_Antediluvian\_Order\_of\_Buffaloes\#:~:text=The%20Royal%20Antediluvian%20Order%20of,to%20members](https://www.wikiwand.com/en/articles/Royal_Antediluvian_Order_of_Buffaloes#:~:text=The%20Royal%20Antediluvian%20Order%20of,to%20members)\)\.AnswerHALF MOON SWIMMING & WATER POLO CLUBChecklist✓The elevated waterway bridge in the picture is the Pontcysyllte Aqueduct\.✓One of the designers of the Pontcysyllte Aqueduct is Thomas Telford\.✓The designer of the Menai Suspension Bridge is Thomas Telford\.✓The Menai Suspension Bridge spans the Menai Strait\.✓The island connected by the Menai Suspension Bridge is the Isle of Anglesey\.✓There is a place called Llanfairpwllgwyngyll on the Isle of Anglesey\.✓Llanfairpwllgwyngyll is popular for its extremely long Welsh place name\.✓The station sign for Llanfairpwllgwyngyll displays this extremely long place name\.✓The railway station with this long name is Llanfairpwll railway station\.✓Llanfairpwll railway station is located on the North Wales Coast Line\.✓The western end of the North Wales Coast Line is Holyhead\.✓Holyhead railway station is located in Holyhead\.×\\timesHolyhead has a ferry route to Dublin\.×\\timesThe Dublin endpoint of the Holyhead–Dublin ferry route is Dublin Port\.×\\timesDublin Port is located on the Dublin Bay side\.×\\timesThe bay entrance related to Dublin Port can be located via the Great South Wall\.×\\timesThe landmark at the end of the Great South Wall is Poolbeg Lighthouse\.×\\timesThe real name of the account The Hiking Hermit Travels is Paul aka\.×\\timesPoolbeg Lighthouse appears in The Hiking Hermit Travels’ video titled “Looking for FREE things to do in DUBLIN? The GREAT SOUTH WALL and POOLBEG LIGHTHOUSE Walk \| 4K video\.”×\\timesThe full name of the club on the house wall in the video is HALF MOON SWIMMING & WATER POLO CLUB\.Is\_Correct0Table 21:Case Study Part 2/3: A late\-chain propagation failure\. Continued\.Table 22:Case Study Part 3/3: An early visual grounding failure\. The model fails checklist items 1–7, guesses item 8 and then infers item 9, but misses the multimodal video step at item 10 and cannot recover item 11\.FieldOriginal contentFigurea1839503![[Uncaptioned image]](https://arxiv.org/html/2609.11318v1/figures/case/a1839503-1.png)QuestionThe image shows a flagship smartphone\. The processor in this phone was developed by a well\-known chip design team, which heavily relies on a leading provider of electronic design automation tools worldwide\. Meanwhile, the metal frame material of this phone was first used in two aircraft\. Of those, the one that was retired later was moved to a museum after completing a mission with the U\.S\. Air Force at Edwards AFB\. This museum is affiliated with a program, which is a department of an institution\. The official YouTube account of the U\.S\. Department of State has posted a video introducing this institution\. In this video, the third person interviewed is wearing a shirt with a few words on it\. What is the first word?Modelopenai/o4\-mini\-deep\-research\-2025\-06\-26ResponseThe smartphone is the Samsung Galaxy S24 Ultra, which uses Samsung’s Exynos chip \(designed using Synopsys EDA tools\) and a titanium frame\. Its frame metal – aerospace\-grade titanium – was first used in two notable aircraft: the Space Shuttle Enterprise \(which flew test landings at Edwards Air Force Base\) and an F\-117 Nighthawk stealth fighter\. The F\-117 \(tail \#783\) was later moved to the Air Force Flight Test Museum at Edwards AFB\. That museum participates in the Smithsonian Affiliations program \(a program of the Smithsonian Institution\)\. The U\.S\. State Department’s YouTube channel has a video introducing the Smithsonian Institution, and in that video the third person interviewed is wearing a shirt with text\. The first word on her shirt is“Smithsonian\.”AnswerclimateChecklist×\\timesThe product displayed in the picture is the Apple iPhone 15 Pro Max×\\timesThe processor chip installed in the iPhone 15 Pro Max is the A17 Pro×\\timesThe A17 Pro chip is designed by the Apple Silicon team×\\timesThe Apple Silicon team collaborates deeply with EDA tool provider Synopsys×\\timesThe frame material of the iPhone 15 Pro Max is titanium×\\timesThe Lockheed A\-12 and the SR\-71 “Blackbird” were two of the first aircraft frames to use titanium×\\timesAfter all USAF and NASA SR\-71 operations at Edwards AFB were completed, the SR\-71 Flight Simulator was moved in July 2006 to the Frontiers of Flight Museum at Love Field Airport in Dallas, Texas✓The Frontiers of Flight Museum is an affiliate within the Smithsonian Affiliations program✓Smithsonian Affiliations is a division of the Smithsonian Institution×\\timesThe official YouTube account of the U\.S\. Department of State has posted a video titled “The Legacy of the Smithsonian Institution”×\\timesIn the video “The Legacy of the Smithsonian Institution,” the first word on the shirt of the third interviewee is “climate”Is\_Correct0Table 22:Case Study Part 3/3: An early visual grounding failure\. Continued\.
## Appendix HRelease, Reproducibility, and Ethics

### H\.1Released Artifacts and Reproducibility

We will release the benchmark data together with the evaluator, the exact judge prompt, verified direct\-dependency graphs, and regression tests\. The benchmark data include the questions, short answers, category/type/subtask labels, irreducible checklists, source URLs, and associated image assets\. These artifacts specify both the evaluation inputs and the scoring procedure, allowing researchers to recompute OA, SA, CS, and DACS from model responses or to re\-judge cached responses under the same protocol\.

For each evaluated model, we record the model identifier, inference configuration, browsing setting, and evaluation outputs used in the analysis\. The regression tests cover score aggregation and dependency\-aware scoring, helping detect implementation changes that would alter reported results\. Together, the data schema, judge prompt, evaluator, and recorded configurations provide a consistent basis for reproducing the evaluation and extending it to new models\.

### H\.2Ethical Safeguards

The benchmark is constructed from publicly accessible web content and is intended to evaluate information seeking, evidence integration, and multimodal reasoning\. During item construction and audit, annotators avoid questions that depend on private or sensitive personal information unless that information is both publicly documented and essential to a legitimate informational task\. Source URLs are retained to support evidence inspection and attribution, and each item is reviewed for answerability, evidential support, and the substantive use of non\-text evidence\.