Deep Research Pretraining via Predictive Navigation

arXiv cs.CL Papers

Summary

Introduces Deep Research Pretraining (DRP), an offline framework that generates search-open-write trajectories from citation and hyperlink evidence structures. Qwen3-14B models pretrained on 1B tokens with DRP outperform matched no-DRP baselines on deep research benchmarks, even with less supervised fine-tuning data.

arXiv:2608.00432v1 Announce Type: new Abstract: Deep research agents are often trained on expensive, environment-grounded tool-use trajectories that require repeated retrieval, document inspection, and report evaluation. We introduce Deep Research Pretraining (DRP), an offline framework that derives predictive navigation supervision from naturally occurring evidence structures. Given a citation-bearing or hyperlinked passage, DRP constructs a proxy research objective, recovers linked evidence and graph-related alternatives, and converts them into search-open-write trajectories. This teaches models what to search for, which documents to inspect, and how to synthesize evidence, without a live retrieval environment or executed policy rollout. We instantiate DRP on scholarly citation graphs (DRP-Paper) and Wikipedia hyperlinks (DRP-Web), continually pretrain separate Qwen3-14B-Base models on 1B tokens, and fine-tune them on controlled fractions of 13K agent trajectories. Across five independently sampled subsets at each low-data budget, both variants consistently outperform matched no-DRP models on DeepResearch Bench. With one quarter of the SFT data, DRP-Web even surpasses a fixed no-DRP full-data checkpoint, with gains transferring to ResearchQA, WebWalkerQA, and SimpleQA. Starting from matched low-data SFT checkpoints, the DRP-Web advantage also persists through subsequent agentic RL. Source-matched and evidence-mismatch controls indicate that these improvements arise from evidence-conditioned navigation rather than domain exposure or agent-format imitation. DRP thus provides a promising complementary approach to trajectory-based agent training.
Original Article
View Cached Full Text

Cached at: 08/04/26, 07:42 AM

# Deep Research Pretraining via Predictive Navigation
Source: [https://arxiv.org/html/2608.00432](https://arxiv.org/html/2608.00432)
###### Abstract

Deep research agents are often trained on expensive, environment\-grounded tool\-use trajectories that require repeated retrieval, document inspection, and report evaluation\. We introduce Deep Research Pretraining \(DRP\), an offline framework that derives predictive\-navigation supervision from naturally occurring evidence structures\. Given a citation\-bearing or hyperlinked passage, DRP constructs a proxy research objective, recovers linked evidence and graph\-related alternatives, and converts them into search–open–write trajectories\. This teaches models what to search for, which documents to inspect, and how to synthesize evidence, without a live retrieval environment or executed policy rollout\. We instantiate DRP on scholarly citation graphs \(DRP\-Paper\) and Wikipedia hyperlinks \(DRP\-Web\), continually pretrain separate Qwen3\-14B\-Base models on 1B tokens, and fine\-tune them on controlled fractions of 13K agent trajectories\. Across five independently sampled subsets at each low\-data budget, both variants consistently outperform matched no\-DRP models on DeepResearch Bench\. With one quarter of the SFT data, DRP\-Web even surpasses a fixed no\-DRP full\-data checkpoint, with gains transferring to ResearchQA, WebWalkerQA, and SimpleQA\. Starting from matched low\-data SFT checkpoints, the DRP\-Web advantage also persists through subsequent agentic RL\. Source\-matched and evidence\-mismatch controls indicate that these improvements arise from evidence\-conditioned navigation rather than domain exposure or agent\-format imitation\. DRP thus provides a promising complementary approach to trajectory\-based agent training\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2608.00432v1/x1.png)Figure 1:DRP overview\. Starting from a target report and its natural evidence graph, DRP infers a proxy research objective, builds search candidate sets from supporting evidence and related alternatives, and synthesizes a history\-conditioned search–open–write trajectory\. Search and open observations are rendered offline from graph neighborhoods and source documents, without a live retrieval environment or policy rollout\.Language\-model agents interleave reasoning with external actions and observations to solve multi\-step tasks\(Yaoet al\.,[2023](https://arxiv.org/html/2608.00432#bib.bib19)\)\. Their capabilities are commonly elicited through post\-training on task\-specific, environment\-grounded trajectories\(Nakanoet al\.,[2021](https://arxiv.org/html/2608.00432#bib.bib18); Shaoet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib2)\)\. Producing such data requires executable environments, long rollouts, repeated tool use, and outcome evaluation, making each usable trajectory costly\. A representative setting in which these costs are amplified is deep research\. A deep\-research agent must reformulate queries, navigate multiple sources, select evidence, recover from unproductive exploration, and synthesize source\-grounded reports\. Recent methods train these abilities using teacher\-generated trajectories or reinforcement learning in real or simulated search environments, often guided by report\-level rubrics\(Jinet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib4); Zhenget al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib5); Sunet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib6); Shaoet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib2)\)\. Although effective, they still depend on costly interaction, trajectory construction, or evaluation\.

Yet document collections already contain evidence structures that arise independently of agent rollouts\. Citation\-bearing and hyperlinked passages associate synthesized text with documents that support it, while neighboring nodes in the citation or hyperlink graph provide related but unselected alternatives\. These relations could potentially supervise the central decisions of deep research: what to search for, which documents to inspect, and how to synthesize the resulting evidence\. Therefore, we ask:*can naturally occurring relations among documents be transformed into rollout\-free pretraining supervision, thereby improving the trajectory\-sample efficiency of downstream agent training?*Our goal is not to replace downstream agent post\-training, but to provide an initialization that reduces its dependence on expensive, environment\-grounded trajectories\.

We introduceDeep Research Pretraining\(DRP\), an offline framework that reverse\-synthesizes such supervision from naturally occurring evidence structures rather than asking a teacher to execute a complete browsing rollout\. Figure[1](https://arxiv.org/html/2608.00432#S1.F1)summarizes the pipeline from natural evidence graphs to predictive\-navigation trajectories\. Starting from a citation\-bearing or hyperlinked passage that serves as a target report, DRP constructs a proxy research objective, recovers linked sources, and draws graph\-related alternatives from the surrounding citation or hyperlink topology\. It then renders these ingredients into search–open–write trajectories\. We call the learning objective*predictive navigation*: conditioned on the research objective and the evidence observed so far, the model learns to predict what to search for, which documents to inspect, and how to synthesize the gathered evidence\. Because candidate sets and document contents are assembled offline from the evidence graph and source corpus, constructing DRP data requires neither a live retrieval environment nor an executed policy rollout\.

We instantiate DRP in two structurally distinct domains\.DRP\-Papertreats citation\-bearing Related Work passages as target reports, infers proxy research objectives from their content, uses resolved cited papers as evidence, and samples topologically nearby but uncited papers as candidate alternatives\.DRP\-Webapplies the same construction to Wikipedia passages: outgoing links whose pages substantively contribute to a passage form its evidence set after filtering incidental mentions, while neighboring nodes in the hyperlink graph provide related alternatives\. Despite these domain\-specific extraction rules, both variants render the resulting supervision with the same search–open–write action space and predictive\-navigation objective\. The graphs are used only during offline data construction; downstream agents operate with standard search and open tools\.

We evaluate DRP as continual pretraining before agent SFT\. Starting from Qwen3\-14B\-Base, we train separate models on 1B tokens of DRP\-Paper or DRP\-Web and then fine\-tune them on controlled fractions of 13K GPT\-5\-generated deep\-research trajectories from DR Tulu\(Shaoet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib2)\)\. Here, sample efficiency is defined as performance as a function of the number of expert trajectories used for downstream training\. Across five independently sampled subsets at both low\-data budgets, DRP\-Paper and DRP\-Web consistently improve over matched no\-DRP SFT on DeepResearch Bench\. With one quarter of the trajectories, DRP\-Web also exceeds a fixed full\-data Base checkpoint\. Positive low\-data gains transfer to ResearchQA, WebWalkerQA, and SimpleQA, and the1/161/16\-SFT effect persists on Qwen3\-30B\-A3B\. Starting from matched1/161/16\-SFT checkpoints, DRP\-Web also remains ahead of Base throughout 80 steps of ARPO\-style agentic RL\. Source\-matched raw\-document and evidence\-mismatch controls support the hypothesis that the gain comes from evidence\-conditioned navigation rather than domain exposure or agent\-format imitation alone\.

Our contributions can be summarized as:

- •We introduce DRP, an offline reverse\-synthesis framework that turns citation and hyperlink relations into history\-conditioned search–open–write supervision without executing an agent rollout\.
- •We instantiate DRP on scholarly citation graphs and the Wikipedia hyperlink graph, and show that a fixed 1B\-token DRP stage consistently improves downstream trajectory\-SFT sample efficiency across independently sampled low\-data subsets\.
- •Source\-matched raw\-CPT and evidence\-mismatch controls support the hypothesis that DRP benefits from evidence\-conditioned navigation rather than domain exposure or agent formatting alone\.

## 2Method

### 2\.1Problem Formulation

Let𝒞=\{dv\}v∈𝒱\\mathcal\{C\}=\\\{d\_\{v\}\\\}\_\{v\\in\\mathcal\{V\}\}be a document corpus equipped with an evidence graph𝒢=\(𝒱,ℰ\)\\mathcal\{G\}=\(\\mathcal\{V\},\\mathcal\{E\}\)\. A directed edge records a naturally occurring relation between two documents, such as a scholarly citation or a Wikipedia hyperlink\. From an anchor document, we extract a citation\-bearing or linked passagerras a target report and recover a set of evidence vertices

Er⊆𝒱E\_\{r\}\\subseteq\\mathcal\{V\}\(1\)whose associated documents\{dv:v∈Er\}\\\{d\_\{v\}:v\\in E\_\{r\}\\\}are referenced by and substantively related to that passage\. We then infer a proxy research objectiveuuwhose answer is expressed byrr\. Lety=Normalize​\(r,Er\)y=\\mathrm\{Normalize\}\(r,E\_\{r\}\)denote the final report target obtained fromrrafter retaining only resolvable evidence\.

A deep\-research trajectory contains both intermediate navigation decisions and a final report\. We write a rendered trajectory as

τ=\(u,\(\(zi,ai,oi\)\)i=1T,zw,y\),\\tau=\\left\(u,\\left\(\(z\_\{i\},a\_\{i\},o\_\{i\}\)\\right\)\_\{i=1\}^\{T\},z\_\{\\mathrm\{w\}\},y\\right\),\(2\)whereziz\_\{i\}is a reasoning trace,aia\_\{i\}is a search or open action,oio\_\{i\}is its observation, andyyis the final report preceded by a write rationalezwz\_\{\\mathrm\{w\}\}\. At decisionii, the model conditions only on the causal history

hi=\(u,\(\(zj,aj,oj\)\)j<i\)h\_\{i\}=\\left\(u,\\left\(\(z\_\{j\},a\_\{j\},o\_\{j\}\)\\right\)\_\{j<i\}\\right\)\(3\)and predicts what to search for, which returned documents to open or ignore, and ultimately how to synthesize the acquired evidence\. We call this learning problem*predictive navigation*\.

### 2\.2Offline Reverse Synthesis

Collectingτ\\tauin the usual forward direction requires executing a policy in a retrieval environment\. DRP instead constructs it in reverse from\(r,Er\)\(r,E\_\{r\}\)\. We partition the evidence targets across search steps,Er=P1∪⋯∪PmE\_\{r\}=P\_\{1\}\\cup\\cdots\\cup P\_\{m\}\. For stepii, the constructor synthesizes a queryqiq\_\{i\}and forms a search candidate set

Ki=Pi∪Ni,K\_\{i\}=P\_\{i\}\\cup N\_\{i\},\(4\)wherePiP\_\{i\}contains evidence vertices to acquire andNiN\_\{i\}contains vertices of topologically related documents that are plausible for the topic but are not selected for the target passage\. The corresponding search observation renders titles and snippets forKiK\_\{i\}\. The open action selectsPiP\_\{i\}and returns their source text; candidates inNiN\_\{i\}are observed but left unopened, providing an implicit selection contrast rather than explicit negative labels\. Section[2\.3](https://arxiv.org/html/2608.00432#S2.SS3)instantiatesErE\_\{r\}and the candidate alternativesNiN\_\{i\}using scholarly citation and Wikipedia hyperlink graphs\.

Our default topology interleaves acquisition and inspection:

S​\(q1\)→O​\(P1\)→⋯→S​\(qm\)→O​\(Pm\)→W​\(y\)\.\\texttt\{S\}\(q\_\{1\}\)\\rightarrow\\texttt\{O\}\(P\_\{1\}\)\\rightarrow\\cdots\\rightarrow\\texttt\{S\}\(q\_\{m\}\)\\rightarrow\\texttt\{O\}\(P\_\{m\}\)\\rightarrow\\texttt\{W\}\(y\)\.\(5\)Consequently, each later query and its synthetic rationale can condition on documents opened earlier\. We generate every rationale using only the information present in its rendered historyhih\_\{i\}, although the offline constructor uses the hidden target report as an oracle for creating the supervision\. We filter rationales that refer to the hidden construction process or unrevealed evidence\. LetCite​\(y\)⊆𝒱\\mathrm\{Cite\}\(y\)\\subseteq\\mathcal\{V\}denote the evidence vertices cited byyy; normalization ensures that every cited evidence document has been opened:

Cite​\(y\)=⋃i=1mPi=Er\.\\mathrm\{Cite\}\(y\)=\\bigcup\_\{i=1\}^\{m\}P\_\{i\}=E\_\{r\}\.\(6\)A batched control moves all open operations after the searches while preserving the underlying queries, evidence, and report; we analyze this topology separately\.

This reverse construction requires no executed agent rollout\. Queries are synthesized from the target contexts, candidate observations come from graph neighborhoods, and open observations are read directly from𝒞\\mathcal\{C\}\. Source links provide weak positive signals, graph neighborhoods provide candidate alternatives, and the source corpus provides offline observations\. All three are obtained without calling a search engine or browser\.

### 2\.3Two Evidence\-Graph Instantiations

#### DRP\-Paper\.

We extract target reports from Related Work passages and resolve their citations to corpus documents through canonical scholarly identifiers\. The resolved cited papers defineErE\_\{r\}, while nearby but uncited papers in the citation graph supply the candidate alternativesNiN\_\{i\}at each search step\. The report target retains only resolvable evidence and uses canonical paper citations\.

#### DRP\-Web\.

We extract target reports from Wikipedia passages and use outgoing links to identify candidate evidence pages\. Because many hyperlinks are incidental mentions, we retain asErE\_\{r\}only linked pages whose content is substantively used by the passage\. Hyperlink\-graph neighborhoods supply the candidate alternativesNiN\_\{i\}\. DRP\-Paper and DRP\-Web thus differ in howErE\_\{r\}andNiN\_\{i\}are obtained, but share the same action space, interleaved construction, and learning objective\.

#### Construction and corpus scale\.

DRP\-Paper is built from the 2024 unarXive snapshot\(Besrour and Färber,[2026](https://arxiv.org/html/2608.00432#bib.bib22)\), while DRP\-Web uses the English FineWiki corpus\(Penedo,[2025](https://arxiv.org/html/2608.00432#bib.bib23)\)\. We use Qwen3\.6\-35B\-A3B\-FP8\(Qwen Team,[2026](https://arxiv.org/html/2608.00432#bib.bib11)\)to infer proxy research objectives, generate search, open, and write rationales, and filter incidental Wikipedia links\. Generation uses a maximum of 32,768 tokens, temperature 0\.7, top\-pp0\.8, and top\-kk20\. Table[1](https://arxiv.org/html/2608.00432#S3.T1)summarizes the resulting corpora\.

### 2\.4Masked Continual\-Pretraining Objective

Letx1:Lx\_\{1:L\}be the tokenized trajectory and letmtm\_\{t\}indicate whether tokenxtx\_\{t\}is produced by the agent\. DRP minimizes the masked causal language\-model loss

ℒDRP​\(θ\)=−1∑t=1Lmt​∑t=1Lmt​log⁡pθ​\(xt∣x<t\)\.\\mathcal\{L\}\_\{\\mathrm\{DRP\}\}\(\\theta\)=\-\\frac\{1\}\{\\sum\_\{t=1\}^\{L\}m\_\{t\}\}\\sum\_\{t=1\}^\{L\}m\_\{t\}\\log p\_\{\\theta\}\(x\_\{t\}\\mid x\_\{<t\}\)\.\(7\)We setmt=1m\_\{t\}=1for assistant reasoning, tool calls and their arguments, and the final report, andmt=0m\_\{t\}=0for the system prompt, user objective, and search/open observations\. The model therefore learns to predict evidence\-conditioned decisions and synthesis rather than to reproduce the constructed environment responses\.

## 3Experiments

![Refer to caption](https://arxiv.org/html/2608.00432v1/x2.png)Figure 2:DeepResearch Bench \(English\) sample efficiency with Qwen3\-14B\.\(a\)Mean scores over five independent SFT subsets; error bars are standard deviations, faint points are individual runs, and full\-SFT points are fixed\.\(b\)Paired score differences from Base at matched SFT fractions \(top\) and fixed full\-SFT Base \(bottom\), with 95%tt\-intervals over subsets\.### 3\.1Experimental Setup

#### Models and continual pretraining\.

Our primary model is Qwen3\-14B\-Base\(Yanget al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib10)\), with Qwen3\-30B\-A3B\-Base used for scale analysis\. Unless stated otherwise, each DRP corpus uses a 1B\-token CPT budget\. We pack 128K\-token sequences with a 2M\-token global batch \(500 steps\) and train in BF16 with AdamW\(Loshchilov and Hutter,[2019](https://arxiv.org/html/2608.00432#bib.bib24)\)\(β1=0\.9\\beta\_\{1\}=0\.9,β2=0\.999\\beta\_\{2\}=0\.999,ϵ=10−8\\epsilon=10^\{\-8\}\), no weight decay, and gradient clipping at 1\.0\. The learning rate has 10% warmup to4×10−54\\times 10^\{\-5\}followed by cosine decay to zero\. A run takes approximately six hours on 128 H20 96GB GPUs\. Canonical DRP\-Web uses the interleaved search–open schedule and medium response budget analyzed in Section[4](https://arxiv.org/html/2608.00432#S4)\.

#### Downstream trajectory SFT\.

Following the DR Tulu SFT recipe\(Shaoet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib2)\), we fine\-tune each initialization for five epochs with 16K\-token sequences, a 2M\-token global batch, and the optimizer settings described above\. We use three data budgets:1/161/16,1/41/4, and all 13,062 English GPT\-5\-generated trajectories from the public release\.222[https://huggingface\.co/datasets/rl\-research/dr\-tulu\-sft\-data](https://huggingface.co/datasets/rl-research/dr-tulu-sft-data)These are environment\-grounded expert demonstrations, so our claim concerns SFT sample efficiency rather than replacing agent training\. At each low\-data fraction, we sample five subsets \(seeds 42, 2026, 5678, 31337, and 1234\); all initializations receive the same trajectories within a seed\. We report mean and standard deviation across runs\. For contrasts involving subset\-trained models, we compute the paired score differencedsd\_\{s\}for each seed and reportd¯±t0\.975,4​sd/5\\bar\{d\}\\pm t\_\{0\.975,4\}s\_\{d\}/\\sqrt\{5\}, wheresds\_\{d\}is the sample standard deviation\. This 95% interval measures SFT\-subset variation; full\-data SFT is a fixed reference\.

#### Downstream agentic RL\.

To test whether DRP remains useful beyond SFT, we initialize two RL runs from Base and DRP\-Web checkpoints trained on the same fixed1/161/16SFT subset\. Both runs follow the ARPO deep\-search setting\(Donget al\.,[2026](https://arxiv.org/html/2608.00432#bib.bib3)\), using the same 1K\-example mixed hard\-search training set and identical rollout, reward, and optimization configurations\. We evaluate the post\-SFT policies before RL and after 40 and 80 optimizer steps\. The reported score is the unweighted mean of ResearchQA coverage, WebWalkerQA accuracy, and SimpleQA accuracy\. Each run takes approximately 24 hours on 64 H20 GPUs\.

#### Evaluation\.

Our main evaluation is the 50\-question English subset of DeepResearch Bench \(DRB\)\(Duet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib1)\)\. We evaluate DRB with its official GPT\-5\.5 RACE evaluator\.333[https://github\.com/Ayanami0730/deep˙research˙bench](https://github.com/Ayanami0730/deep_research_bench)Agents use the DR\-Tulu protocol, Serper search, and Jina extraction, with at most 20 tool\-call rounds and 32K output tokens\. Decoding uses temperature 0\.6, top\-pp0\.95, and top\-kk20\. We also report report\-generation rate and mean search/open calls\. Score differences are absolute points on this scale, not relative percentage changes\. For single\-checkpoint controls, we jointly resample question IDs, recompute the paired score difference, and take its 2\.5th and 97\.5th percentiles; unlike thett\-intervals above, these paired\-bootstrap intervals measure question uncertainty\(Efron and Tibshirani,[1994](https://arxiv.org/html/2608.00432#bib.bib25)\)\. For transfer, we evaluate fixed sets of 500 examples from ResearchQA\(Li S\. Yifeiet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib15)\), WebWalkerQA\(Wuet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib16)\), and SimpleQA\(Weiet al\.,[2024](https://arxiv.org/html/2608.00432#bib.bib17)\)with the same agent, reporting coverage for ResearchQA and accuracy otherwise\.

Table 1:DRP corpus statistics before subsampling to the matched 1B\-token CPT budget\. Available is the total token count; Loss % is the fraction included in the language\-model loss\.

### 3\.2Main Results

#### DRP improves trajectory\-SFT sample efficiency\.

Figure[2](https://arxiv.org/html/2608.00432#S3.F2)gives the central result\. Across five independently sampled subsets at both1/161/16and1/41/4SFT, DRP\-Paper and DRP\-Web consistently improve over directly fine\-tuning the base model\. With only one quarter of the expert trajectories, DRP\-Web exceeds the no\-DRP full\-data model by 1\.0 point, with its 95% interval above zero\. DRP\-Paper has a positive mean difference of 0\.7 points, although its interval overlaps zero\. The gain is largest in the lowest\-data regime, where DRP also substantially reduces empty\-report failures and produces a functioning tool\-use policy\. Figure[2](https://arxiv.org/html/2608.00432#S3.F2)\(b\) summarizes both matched\-subset gains and direct comparisons with the fixed full\-SFT Base model\. The positive same\-fraction gains show that the result recurs across independently sampled low\-data training sets rather than depending on one selected subset\. Appendix[D](https://arxiv.org/html/2608.00432#A4)estimates at least 51\.8% lower standardized data\-construction cost\.

![Refer to caption](https://arxiv.org/html/2608.00432v1/x3.png)Figure 3:RACE dimension scores\. Low\-data bars average five independently sampled, matched SFT subsets\.
#### The gain spans all RACE dimensions\.

Figure[3](https://arxiv.org/html/2608.00432#S3.F3)decomposes the score into comprehensiveness, insight, instruction following, and readability\. At both low\-data fractions, DRP\-Paper and DRP\-Web improve every dimension over Base\. The gains are distributed across content\-oriented dimensions \(comprehensiveness and insight\) as well as instruction following and readability, rather than being concentrated in a single aspect of report generation\.

#### The gain transfers beyond long\-form report evaluation\.

Figure[4](https://arxiv.org/html/2608.00432#S3.F4)shows that the benefit is not confined to long\-form report evaluation\. At both1/161/16and1/41/4SFT, DRP\-Paper and DRP\-Web have positive paired gains over Base on ResearchQA coverage, WebWalkerQA accuracy, and SimpleQA accuracy\. Under full\-data SFT, the effects become small and task dependent, with most confidence intervals overlapping zero\. This attenuation is consistent with DRP acting as a sample\-efficient initialization rather than directly optimizing each downstream benchmark\.

![Refer to caption](https://arxiv.org/html/2608.00432v1/x4.png)Figure 4:Transfer over Base on 500 examples per benchmark\. Markers show paired metric differences at matched SFT fractions; the metric is coverage for ResearchQA and accuracy otherwise\. Bars are 95% paired\-bootstrap intervals over questions\.![Refer to caption](https://arxiv.org/html/2608.00432v1/x5.png)Figure 5:Agentic RL from matched1/161/16\-SFT checkpoints\. Both initializations follow the same ARPO deep\-search setting\. Scores are the macro\-average of ResearchQA coverage, WebWalkerQA accuracy, and SimpleQA accuracy\.
#### The advantage persists through agentic RL\.

Figure[5](https://arxiv.org/html/2608.00432#S3.F5)tests whether the stronger DRP initialization persists under subsequent policy optimization\. Before RL, DRP\-Web leads Base by 15\.0 points \(57\.2 versus 42\.2\)\. The gap narrows but remains 6\.8 points after 40 steps and 4\.8 points after 80 steps\. Base after 80 steps still does not reach the pre\-RL DRP\-Web checkpoint \(54\.4 versus 57\.2\), while DRP\-Web itself improves to 59\.2\. Thus, DRP provides a stronger initial policy that remains advantageous under a fixed downstream RL budget and remains compatible with further policy optimization\.

## 4Analysis

Unless stated otherwise, analysis experiments use Qwen3\-14B, a 1B\-token CPT budget, and the same fixed1/161/16DR Tulu SFT subset; DRP\-Web is the default corpus when no other corpus is named\. All controls use the CPT and SFT optimization hyperparameters from Section[3](https://arxiv.org/html/2608.00432#S3), including sequence lengths, global batch sizes, learning\-rate schedules, and SFT epochs\. Thus, each comparison changes only the factor named by the ablation while holding the training budget and downstream supervision fixed\.

### 4\.1Raw Domain Pretraining Is Not Enough

We first test whether DRP’s gain can be explained by exposure to the underlying source domains\. Raw\-Web and Raw\-Paper train directly on the same source documents used to construct their respective DRP corpora, but contain no proxy queries, tool calls, or graph\-derived relevance relations\. Within each domain, we compare Base, raw\-document CPT, and structured DRP under the same 1B\-token CPT budget and downstream SFT setting\. In Figure[6](https://arxiv.org/html/2608.00432#S4.F6)\(a\), DRP\-Web gains 5\.3 points over Base, compared with 0\.8 for Raw\-Web; DRP\-Paper gains 5\.0 points, whereas Raw\-Paper decreases by 4\.8\. The corresponding direct DRP–Raw advantages are 4\.5 and 9\.8 points, with both paired\-bootstrap intervals above zero \(Figure[6](https://arxiv.org/html/2608.00432#S4.F6)\(b\)\)\. Thus, source\-matched domain exposure is not sufficient to account for the predictive\-navigation gain\.

![Refer to caption](https://arxiv.org/html/2608.00432v1/x6.png)Figure 6:Raw\-domain CPT controls on DeepResearch Bench \(English; Qwen3\-14B; 1B\-token CPT;1/161/16SFT\)\.\(a\)Paired score differences from Base\.\(b\)DRP–Raw score differences with domain\-matched source documents\. Bars are 95% paired\-bootstrap intervals over questions\.
### 4\.2Evidence Alignment Matters

The raw\-document controls establish an advantage for structured DRP over raw CPT, but do not isolate whether the constructed observations must be relevant to the model’s decisions\. We therefore perform a controlled ablation in our canonical DRP\-Web analysis setting\. Evidence\-Mismatched DRP\-Web preserves the proxy queries, synthetic reasoning, action schedule, tool\-call syntax, response\-length distribution, and training budget of aligned DRP\-Web, but replaces graph\-derived search and open observations with random in\-domain evidence\. Aligned DRP\-Web gains 5\.3 points over Base, while evidence\-mismatched DRP\-Web gains only 1\.1; their direct difference is 4\.2 points with its 95% bootstrap interval above zero \(Figure[7](https://arxiv.org/html/2608.00432#S4.F7)\)\. Breaking query–observation–action alignment therefore removes most of the gain even though the agent\-like surface form is preserved\.

![Refer to caption](https://arxiv.org/html/2608.00432v1/x7.png)Figure 7:Evidence\-alignment control on DeepResearch Bench \(English; Qwen3\-14B; 1B\-token DRP\-Web CPT; fixed1/161/16\-SFT subset\)\.\(a\)Paired score differences from Base for aligned and evidence\-mismatched DRP\-Web\.\(b\)Their direct score difference\. Bars are 95% paired\-bootstrap intervals over questions\.
### 4\.3Which Pseudo\-operations Matter?

Table 2:DRP\-Web design ablations on DeepResearch Bench \(English; Qwen3\-14B; 1B\-token CPT; fixed1/161/16\-SFT subset\)\. Reports is the valid\-report rate, and Calls the mean search/open calls per question\.SSSOWandSOSOWare mnemonic topology names rather than fixed\-length traces\.†denotes canonical choices; bold marks each block’s best score\.LetS,O, andWdenote search, open, and write\. The namesSSSOWandSOSOWdenote topology rather than a fixed operation count: the former contains a variable number of searches followed by open and write, while the latter contains a variable number of search–open rounds followed by write\. Table[2](https://arxiv.org/html/2608.00432#S4.T2)compares these constructions with search\-only traces\. Search\-only reaches 36\.4, 3\.4 points above no CPT \(33\.0\)\. BatchedSSSOWreaches 37\.3, and interleavedSOSOWreaches 38\.3 while using fewer downstream calls than the batched schedule \(5\.7 versus 6\.7\)\. This 1\.0\-point advantage is consistent with feedback\-conditioned navigation, although the single fixed\-subset comparison does not isolate every difference in the constructed histories\.

### 4\.4Protocol Rendering

We render the same DRP records in either the compact DR\-Tulu protocol used downstream or a multi\-turn Hermes protocol that alternates assistant actions and tool observations\. The underlying evidence, think traces, and tool calls are unchanged\. Matching the downstream DR\-Tulu protocol raises the score from 34\.0 to 38\.3 and report rate from 86% to 100% \(Table[2](https://arxiv.org/html/2608.00432#S4.T2)\)\. Protocol alignment therefore materially affects transfer efficiency\. It is not sufficient to explain the full DRP gain, however, because evidence mismatch reduces the matched\-format model’s gain over Base from 5\.3 to 1\.1 points \(Figure[7](https://arxiv.org/html/2608.00432#S4.F7)\)\.

### 4\.5Tool Response Length

Tool observations are masked from the language\-model loss\. Shorter responses therefore reduce training tokens and increase the fraction of supervised tokens, but may omit evidence needed for grounded decisions\. Under the fixed1/161/16setting, the mid budget reaches a score of 38\.3 with a 100% report rate, compared with 36\.4/96% for short and 35\.6/92% for long \(Table[2](https://arxiv.org/html/2608.00432#S4.T2)\)\. Longer context therefore provides no monotonic benefit in this experiment\. We use mid as the default practical tradeoff, but do not interpret this single\-setting ordering as a universal optimum across downstream SFT budgets\.

### 4\.6Transfer Across Model Scales

We repeat the1/161/16\-SFT DRP\-Paper and DRP\-Web comparisons on Qwen3\-30B\-A3B\-Base\. Relative to matched no\-CPT models, DRP\-Paper and DRP\-Web gain 5\.0 and 5\.3 points at 14B, and 4\.7 and 3\.1 points at 30B\-A3B \(Table[3](https://arxiv.org/html/2608.00432#S4.T3)\)\. The positive gains for both evidence graphs at both scales support transfer of the low\-data effect; they do not establish a monotonic model\-scaling law\.

Table 3:Model\-scale transfer on DeepResearch Bench \(English\)\. Values are score gains from 1B\-token DRP over no\-CPT Base after matched SFT on one fixed1/161/16subset; hence Qwen3\-14B values differ from the five\-subset means in Figure[2](https://arxiv.org/html/2608.00432#S3.F2)\.
### 4\.7General Capability and Distribution Shift

We test whether specializing CPT toward DRP harms general capabilities under a fixed 2B\-token budget starting from Qwen3\-14B\-Base\. The control uses 2B tokens of general midtraining data from the fully open OLMo 3 training flow\(Team OLMoet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib12)\); the mixed condition replaces half of that budget with 1B DRP\-Web tokens\. Our 14\-benchmark suite spans commonsense reasoning \(e\.g\., ARC\), mathematical reasoning \(GSM8K\), knowledge \(MMLU\-Redux\), and reading comprehension \(DROP\); Appendix[C\.4](https://arxiv.org/html/2608.00432#A3.SS4)provides the complete suite, citations, and per\-benchmark results\. Table[4](https://arxiv.org/html/2608.00432#S4.T4)shows that the aggregate score changes from 73\.1 to 73\.4\. After matched1/161/16DR\-Tulu SFT, the mixed checkpoint improves the score from 33\.1 to 36\.8 \(\+3\.7 points\)\. Thus, replacing half of the general CPT budget with DRP\-Web preserves aggregate general capability in this evaluation while improving downstream agent initialization\.

Table 4:General capability and downstream performance under a fixed 2B\-token CPT budget\. General average is measured after CPT; DRB is measured after matched1/161/16SFT\.
### 4\.8Masking Tool Observations Is Important

Our default objective computes loss only on assistant reasoning, tool calls and arguments, and the final report; tool observations are context\. A no\-mask control additionally trains the model to predict the constructed search and open responses\. Removing the mask lowers the score from 38\.3 to 30\.2, below the no\-CPT score of 33\.0, and reduces mean downstream tool calls from 5\.7 to 0\.6 \(Table[2](https://arxiv.org/html/2608.00432#S4.T2)\)\. This drop of 8\.1 points shows that predicting constructed observations is detrimental in this setting; masking them focuses the objective on agent decisions and synthesis\.

## 5Related Work

#### Training and supervising research agents\.

WebGPT and ReAct establish browser\-assisted question answering and the interleaving of language\-model reasoning with external actions\(Nakanoet al\.,[2021](https://arxiv.org/html/2608.00432#bib.bib18); Yaoet al\.,[2023](https://arxiv.org/html/2608.00432#bib.bib19)\)\. Recent deep\-research systems train these behaviors through search\-environment RL, teacher\-generated trajectories, synthetic tasks, and report\-level rubrics\(Jinet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib4); Zhenget al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib5); Shaoet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib2); Xieet al\.,[2026](https://arxiv.org/html/2608.00432#bib.bib13)\)\. ARPO further adapts policy optimization to multi\-turn tool\-use exploration\(Donget al\.,[2026](https://arxiv.org/html/2608.00432#bib.bib3)\)\. ZeroSearch and OpenResearcher reduce the cost of this supervision through simulated retrieval or a local search\-and\-browse environment\(Sunet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib6); Liet al\.,[2026](https://arxiv.org/html/2608.00432#bib.bib7)\)\. These approaches improve the downstream post\-training process but still obtain navigation supervision from executed or simulated trajectories\. DRP is complementary: it constructs an offline predictive\-navigation initialization from existing evidence graphs before trajectory SFT or RL\.

#### Continual and synthetic pretraining\.

Domain\-adaptive pretraining shows that continued training on in\-domain text can improve downstream adaptation\(Gururanganet al\.,[2020](https://arxiv.org/html/2608.00432#bib.bib20)\); more recent work extends continual pretraining to agent behaviors or restructures source documents into higher\-utility synthetic data\(Suet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib8); Niklauset al\.,[2026](https://arxiv.org/html/2608.00432#bib.bib14)\)\. WRAP\+\+ uses hyperlink motifs to synthesize cross\-document QA\(Zhouet al\.,[2026](https://arxiv.org/html/2608.00432#bib.bib40)\)\. These literature motivates both our pre\-SFT intervention and our source\-matched raw\-CPT controls\. DRP differs by converting document relations into history\-conditioned evidence\-acquisition decisions rather than only exposing the model to domain text or synthesizing cross\-document knowledge\.

#### Weak supervision from document structure\.

Naturally occurring document context has been used to generate pseudoqueries for weakly supervised retriever pretraining, while retrieved evidence structure can support dynamic planning at inference time\(Leeet al\.,[2019](https://arxiv.org/html/2608.00432#bib.bib21); Liet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib9)\)\. DRP connects these ideas at a different stage: citation and hyperlink relations provide offline supervision for an autoregressive policy spanning repeated search, evidence inspection, and report synthesis\. Thus, the graph is neither only a single\-step retrieval target nor an inference\-time memory\.

## 6Conclusion

DRP turns naturally occurring citation and hyperlink relations into offline predictive\-navigation supervision before downstream agent training\. It renders graph\-derived evidence and alternatives into history\-conditioned search, open, and write decisions without live browser rollouts\. Across five matched low\-data subsets, both corpora improve trajectory sample efficiency; DRP\-Web with one\-quarter SFT surpasses the full\-data no\-DRP Base\. This recipe uses 75% fewer expert trajectories and reduces the standardized estimate of data\-construction cost by at least 51\.8% \(Appendix[D](https://arxiv.org/html/2608.00432#A4)\)\. Gains transfer to other benchmarks, persist through agentic RL, and span evidence graphs and model scales\. Source\-matched raw\-CPT and evidence\-mismatch controls attribute the benefit to navigation alignment rather than domain exposure or trace formatting, while general\-data mixing preserves aggregate capability\. DRP is therefore a complement, not a substitute, for expert\-trajectory SFT and RL: it moves evidence acquisition into initialization so each downstream trajectory has greater value\. This matters when environments are hard to deploy, feedback is delayed, or expert traces cover few workflows\. Our evidence remains limited to English deep\-research tasks, and the gains narrow with more supervision\. Future work should test other evidence\-bearing graphs and long\-latency environments, and study how graph quality and alternative selection shape the benefits\.

## References

- I\. Besrour and M\. Färber \(2026\)UnarXive 2024: a large\-scale scientific corpus for citation\-aware retrieval and generation\.InProceedings of the Fifteenth Language Resources and Evaluation Conference \(LREC 2026\),LREC 2026,Palma, Mallorca, Spain,pp\. 6990–6997\.External Links:[Document](https://dx.doi.org/10.63317/2nqzwzhq3j3t),ISSN 2522\-2686,ISBN 978\-2\-493814\-49\-4Cited by:[§A\.1](https://arxiv.org/html/2608.00432#A1.SS1.SSS0.Px1.p1.1),[§2\.3](https://arxiv.org/html/2608.00432#S2.SS3.SSS0.Px3.p1.2)\.
- Y\. Bisk, R\. Zellers, R\. Le Bras, J\. Gao, and Y\. Choi \(2020\)PIQA: reasoning about physical commonsense in natural language\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.34,pp\. 7432–7439\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v34i05.6239)Cited by:[§C\.4](https://arxiv.org/html/2608.00432#A3.SS4.p1.1)\.
- P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. Tafjord \(2018\)Think you have solved question answering? try ARC, the AI2 reasoning challenge\.External Links:1803\.05457Cited by:[§C\.4](https://arxiv.org/html/2608.00432#A3.SS4.p1.1)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman \(2021\)Training verifiers to solve math word problems\.External Links:2110\.14168Cited by:[§C\.4](https://arxiv.org/html/2608.00432#A3.SS4.p1.1)\.
- G\. Dong, H\. Mao, K\. Ma, L\. Bao, Y\. Chen, Z\. Wang, Z\. Chen, J\. Du, H\. Wang, F\. Zhang, G\. Zhou, Y\. Zhu, J\. Wen, and Z\. Dou \(2026\)Agentic reinforced policy optimization\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=TX4k7BF6aO)Cited by:[§A\.5](https://arxiv.org/html/2608.00432#A1.SS5.p1.2),[§3\.1](https://arxiv.org/html/2608.00432#S3.SS1.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2608.00432#S5.SS0.SSS0.Px1.p1.1)\.
- M\. Du, B\. Xu, C\. Zhu, X\. Wang, and Z\. Mao \(2025\)DeepResearch bench: a comprehensive benchmark for deep research agents\.External Links:2506\.11763Cited by:[§A\.6](https://arxiv.org/html/2608.00432#A1.SS6.p2.1),[§3\.1](https://arxiv.org/html/2608.00432#S3.SS1.SSS0.Px4.p1.3)\.
- D\. Dua, Y\. Wang, P\. Dasigi, G\. Stanovsky, S\. Singh, and M\. Gardner \(2019\)DROP: a reading comprehension benchmark requiring discrete reasoning over paragraphs\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics,pp\. 2368–2378\.External Links:[Document](https://dx.doi.org/10.18653/v1/N19-1246)Cited by:[§C\.4](https://arxiv.org/html/2608.00432#A3.SS4.p1.1)\.
- B\. Efron and R\. J\. Tibshirani \(1994\)An introduction to the bootstrap\.Chapman and Hall/CRC\.Cited by:[§3\.1](https://arxiv.org/html/2608.00432#S3.SS1.SSS0.Px4.p1.3)\.
- A\. P\. Gemaet al\.\(2025\)Are we done with MMLU?\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics,pp\. 5069–5096\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.262)Cited by:[§C\.4](https://arxiv.org/html/2608.00432#A3.SS4.p1.1)\.
- S\. Gururangan, A\. Marasović, S\. Swayamdipta, K\. Lo, I\. Beltagy, D\. Downey, and N\. A\. Smith \(2020\)Don’t stop pretraining: adapt language models to domains and tasks\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,pp\. 8342–8360\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.740)Cited by:[§5](https://arxiv.org/html/2608.00432#S5.SS0.SSS0.Px2.p1.1)\.
- X\. Ho, A\. D\. Nguyen, S\. Sugawara, and A\. Aizawa \(2020\)Constructing a multi\-hop QA dataset for comprehensive evaluation of reasoning steps\.InProceedings of the 28th International Conference on Computational Linguistics,pp\. 6609–6625\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.coling-main.580)Cited by:[§C\.4](https://arxiv.org/html/2608.00432#A3.SS4.p1.1)\.
- B\. Jin, H\. Zeng, Z\. Yue, D\. Wang, H\. Zamani, and J\. Han \(2025\)Search\-r1: training llms to reason and leverage search engines with reinforcement learning\.External Links:2503\.09516Cited by:[§1](https://arxiv.org/html/2608.00432#S1.p1.1),[§5](https://arxiv.org/html/2608.00432#S5.SS0.SSS0.Px1.p1.1)\.
- M\. Joshi, E\. Choi, D\. Weld, and L\. Zettlemoyer \(2017\)TriviaQA: a large scale distantly supervised challenge dataset for reading comprehension\.InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics,pp\. 1601–1611\.External Links:[Document](https://dx.doi.org/10.18653/v1/P17-1147)Cited by:[§C\.4](https://arxiv.org/html/2608.00432#A3.SS4.p1.1)\.
- K\. Lee, M\. Chang, and K\. Toutanova \(2019\)Latent retrieval for weakly supervised open domain question answering\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,pp\. 6086–6096\.External Links:[Document](https://dx.doi.org/10.18653/v1/P19-1612)Cited by:[§5](https://arxiv.org/html/2608.00432#S5.SS0.SSS0.Px3.p1.1)\.
- Li S\. Yifei, A\. Chang, C\. Malaviya, and M\. Yatskar \(2025\)ResearchQA: evaluating scholarly question answering at scale across 75 fields with survey\-mined questions and rubrics\.External Links:2509\.00496Cited by:[§A\.6](https://arxiv.org/html/2608.00432#A1.SS6.p3.1),[§3\.1](https://arxiv.org/html/2608.00432#S3.SS1.SSS0.Px4.p1.3)\.
- Z\. Li, D\. Jiang, X\. Ma, H\. Zhang, P\. Nie, Y\. Zhang, K\. Zou, J\. Xie, Y\. Zhang, and W\. Chen \(2026\)OpenResearcher: a fully open pipeline for long\-horizon deep research trajectory synthesis\.External Links:2603\.20278Cited by:[§5](https://arxiv.org/html/2608.00432#S5.SS0.SSS0.Px1.p1.1)\.
- Z\. Li, X\. Guan, B\. Zhang, S\. Huang, H\. Zhou, S\. Lai, M\. Yan, Y\. Jiang, P\. Xie, F\. Huang, J\. Zhang, and J\. Zhou \(2025\)WebWeaver: structuring web\-scale evidence with dynamic outlines for open\-ended deep research\.External Links:2509\.13312Cited by:[§5](https://arxiv.org/html/2608.00432#S5.SS0.SSS0.Px3.p1.1)\.
- I\. Loshchilov and F\. Hutter \(2019\)Decoupled weight decay regularization\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Bkg6RiCqY7)Cited by:[§3\.1](https://arxiv.org/html/2608.00432#S3.SS1.SSS0.Px1.p1.4)\.
- T\. Mihaylov, P\. Clark, T\. Khot, and A\. Sabharwal \(2018\)Can a suit of armor conduct electricity? a new dataset for open book question answering\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,pp\. 2381–2391\.External Links:[Document](https://dx.doi.org/10.18653/v1/D18-1260)Cited by:[§C\.4](https://arxiv.org/html/2608.00432#A3.SS4.p1.1)\.
- R\. Nakano, J\. Hilton, S\. Balaji, J\. Wu, L\. Ouyang, C\. Kim, C\. Hesse, S\. Jain, V\. Kosaraju, W\. Saunders, X\. Jiang, K\. Cobbe, T\. Eloundou, G\. Krueger, K\. Button, M\. Knight, B\. Chess, and J\. Schulman \(2021\)WebGPT: browser\-assisted question\-answering with human feedback\.External Links:2112\.09332Cited by:[§1](https://arxiv.org/html/2608.00432#S1.p1.1),[§5](https://arxiv.org/html/2608.00432#S5.SS0.SSS0.Px1.p1.1)\.
- J\. Niklaus, A\. Yamaguchi, M\. Štefánik, G\. Penedo, H\. Kydlíček, E\. Bakouch, L\. Tunstall, E\. E\. Beeching, T\. Frere, C\. Raffel, L\. von Werra, and T\. Wolf \(2026\)How can we synthesize high\-quality pretraining data? a systematic study of prompt design, generator model, and source data\.External Links:2604\.13977Cited by:[§5](https://arxiv.org/html/2608.00432#S5.SS0.SSS0.Px2.p1.1)\.
- P\. Pasupat and P\. Liang \(2015\)Compositional semantic parsing on semi\-structured tables\.InProceedings of the 53rd Annual Meeting of the Association for Computational Linguistics,pp\. 1470–1480\.External Links:[Document](https://dx.doi.org/10.3115/v1/P15-1142)Cited by:[§C\.4](https://arxiv.org/html/2608.00432#A3.SS4.p1.1)\.
- G\. Penedo \(2025\)FineWiki\.Note:Hugging Face DatasetsExternal Links:[Link](https://huggingface.co/datasets/HuggingFaceFW/finewiki)Cited by:[§A\.1](https://arxiv.org/html/2608.00432#A1.SS1.SSS0.Px1.p1.1),[§2\.3](https://arxiv.org/html/2608.00432#S2.SS3.SSS0.Px3.p1.2)\.
- Qwen Team \(2026\)Qwen3\.6\-35B\-A3B: agentic coding power, now open to all\.External Links:[Link](https://qwen.ai/blog?id=qwen3.6-35b-a3b)Cited by:[§A\.1](https://arxiv.org/html/2608.00432#A1.SS1.SSS0.Px3.p1.2),[§2\.3](https://arxiv.org/html/2608.00432#S2.SS3.SSS0.Px3.p1.2)\.
- P\. Rajpurkar, R\. Jia, and P\. Liang \(2018\)Know what you don’t know: unanswerable questions for SQuAD\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics,pp\. 784–789\.External Links:[Document](https://dx.doi.org/10.18653/v1/P18-2124)Cited by:[§C\.4](https://arxiv.org/html/2608.00432#A3.SS4.p1.1)\.
- R\. Shao, A\. Asai, S\. Z\. Shen, H\. Ivison, V\. Kishore, J\. Zhuo, X\. Zhao, M\. Park, S\. G\. Finlayson, D\. Sontag, T\. Murray, S\. Min, P\. Dasigi, L\. Soldaini, F\. Brahman, W\. Yih, T\. Wu, L\. Zettlemoyer, Y\. Kim, H\. Hajishirzi, and P\. W\. Koh \(2025\)DR tulu: reinforcement learning with evolving rubrics for deep research\.External Links:2511\.19399Cited by:[§A\.4](https://arxiv.org/html/2608.00432#A1.SS4.p1.2),[§1](https://arxiv.org/html/2608.00432#S1.p1.1),[§1](https://arxiv.org/html/2608.00432#S1.p5.2),[§3\.1](https://arxiv.org/html/2608.00432#S3.SS1.SSS0.Px2.p1.5),[§5](https://arxiv.org/html/2608.00432#S5.SS0.SSS0.Px1.p1.1)\.
- L\. Su, Z\. Zhang, G\. Li, Z\. Chen, C\. Wang, M\. Song, X\. Wang, K\. Li, J\. Wu, X\. Chen, Z\. Qiao, Z\. Zhang, H\. Yin, S\. Cai, R\. Fang, Z\. Tao, W\. Yin, C\. Qian, Y\. Jiang, P\. Xie, F\. Huang, and J\. Zhou \(2025\)Scaling agents via continual pre\-training\.External Links:2509\.13310Cited by:[§5](https://arxiv.org/html/2608.00432#S5.SS0.SSS0.Px2.p1.1)\.
- H\. Sun, Z\. Qiao, J\. Guo, X\. Fan, Y\. Hou, Y\. Jiang, P\. Xie, F\. Huang, and Y\. Zhang \(2025\)ZeroSearch: incentivize the search capability of llms without searching\.External Links:2505\.04588Cited by:[§1](https://arxiv.org/html/2608.00432#S1.p1.1),[§5](https://arxiv.org/html/2608.00432#S5.SS0.SSS0.Px1.p1.1)\.
- A\. Talmor, J\. Herzig, N\. Lourie, and J\. Berant \(2019\)CommonsenseQA: a question answering challenge targeting commonsense knowledge\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics,pp\. 4149–4158\.External Links:[Document](https://dx.doi.org/10.18653/v1/N19-1421)Cited by:[§C\.4](https://arxiv.org/html/2608.00432#A3.SS4.p1.1)\.
- Team OLMo, A\. Ettinger, A\. Bertsch,et al\.\(2025\)OLMo 3\.External Links:2512\.13961Cited by:[§4\.7](https://arxiv.org/html/2608.00432#S4.SS7.p1.1)\.
- H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. Sabharwal \(2022\)MuSiQue: multihop questions via single\-hop question composition\.Transactions of the Association for Computational Linguistics10,pp\. 539–554\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00475)Cited by:[§C\.4](https://arxiv.org/html/2608.00432#A3.SS4.p1.1)\.
- J\. Wei, K\. Nguyen, H\. W\. Chung, Y\. J\. Jiao, S\. Papay, A\. Glaese, J\. Schulman, and W\. Fedus \(2024\)Measuring short\-form factuality in large language models\.External Links:2411\.04368Cited by:[§A\.6](https://arxiv.org/html/2608.00432#A1.SS6.p3.1),[§3\.1](https://arxiv.org/html/2608.00432#S3.SS1.SSS0.Px4.p1.3)\.
- J\. Wu, W\. Yin, Y\. Jiang, Z\. Wang, Z\. Xi, R\. Fang, L\. Zhang, Y\. He, D\. Zhou, P\. Xie, and F\. Huang \(2025\)WebWalker: benchmarking LLMs in web traversal\.External Links:2501\.07572Cited by:[§A\.6](https://arxiv.org/html/2608.00432#A1.SS6.p3.1),[§3\.1](https://arxiv.org/html/2608.00432#S3.SS1.SSS0.Px4.p1.3)\.
- J\. Xie, T\. Lin, Z\. Wang, Y\. Ning, Y\. Yao, T\. Xue, Z\. Zhang, Z\. Li, K\. Zhang, Y\. Wu, S\. Chen, B\. Gou, M\. Han, Y\. Wang, V\. Lee, X\. Wei, X\. Wang, Y\. Su, and H\. Sun \(2026\)QUEST: training frontier deep research agents with fully synthetic tasks\.External Links:2605\.24218Cited by:[§5](https://arxiv.org/html/2608.00432#S5.SS0.SSS0.Px1.p1.1)\.
- A\. Yang, A\. Li,et al\.\(2025\)Qwen3 technical report\.External Links:2505\.09388Cited by:[§3\.1](https://arxiv.org/html/2608.00432#S3.SS1.SSS0.Px1.p1.4)\.
- Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. W\. Cohen, R\. Salakhutdinov, and C\. D\. Manning \(2018\)HotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,pp\. 2369–2380\.External Links:[Document](https://dx.doi.org/10.18653/v1/D18-1259)Cited by:[§C\.4](https://arxiv.org/html/2608.00432#A3.SS4.p1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2023\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=WE_vluYUL-X)Cited by:[§1](https://arxiv.org/html/2608.00432#S1.p1.1),[§5](https://arxiv.org/html/2608.00432#S5.SS0.SSS0.Px1.p1.1)\.
- R\. Zellers, A\. Holtzman, Y\. Bisk, A\. Farhadi, and Y\. Choi \(2019\)HellaSwag: can a machine really finish your sentence?\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,pp\. 4791–4800\.External Links:[Document](https://dx.doi.org/10.18653/v1/P19-1472)Cited by:[§C\.4](https://arxiv.org/html/2608.00432#A3.SS4.p1.1)\.
- Y\. Zheng, D\. Fu, X\. Hu, X\. Cai, L\. Ye, P\. Lu, and P\. Liu \(2025\)DeepResearcher: scaling deep research via reinforcement learning in real\-world environments\.External Links:2504\.03160Cited by:[§1](https://arxiv.org/html/2608.00432#S1.p1.1),[§5](https://arxiv.org/html/2608.00432#S5.SS0.SSS0.Px1.p1.1)\.
- J\. Zhou, Y\. Wang, X\. Wu, T\. Yu, and F\. Zhang \(2026\)WRAP\+\+: web discovery amplified pretraining\.External Links:2604\.06829,[Link](https://arxiv.org/abs/2604.06829)Cited by:[§A\.1](https://arxiv.org/html/2608.00432#A1.SS1.SSS0.Px2.p1.3),[§5](https://arxiv.org/html/2608.00432#S5.SS0.SSS0.Px2.p1.1)\.

## Appendix AReproducibility Details

This appendix provides implementation details, statistical procedures, numerical results, and precise definitions of the analysis controls\.

### A\.1DRP Data Construction

#### Source corpora\.

DRP\-Paper is constructed from the 2024 unarXive snapshot\(Besrour and Färber,[2026](https://arxiv.org/html/2608.00432#bib.bib22)\)\. DRP\-Web is constructed from the English FineWiki corpus and its Wikipedia hyperlink structure\(Penedo,[2025](https://arxiv.org/html/2608.00432#bib.bib23)\)\. After filtering, DRP\-Paper contains 78,557 trajectories and 1\.99B available tokens; DRP\-Web contains 8,997,455 trajectories and 172\.46B available tokens\. Token statistics use the Qwen3\-8B tokenizer\. All matched comparisons in the main paper subsample each DRP corpus to a 1B\-token continual\-pretraining budget\.

#### Candidate alternatives\.

For DRP\-Paper, at each search round we construct the alternative pool from the forward\-citation neighbors of the evidence papers assigned to that round\. We exclude papers in the target evidence set and filter highly cited graph hubs, which are often generic references \(e\.g\., AdamW\) rather than topic\-specific alternatives\. For DRP\-Web, we use the dual\-link \(u↔vu\\leftrightarrow v\) and co\-mention \(u→e←vu\\rightarrow e\\leftarrow vwithu→vu\\rightarrow v\) relations following WRAP\+\+\(Zhouet al\.,[2026](https://arxiv.org/html/2608.00432#bib.bib40)\)\. Pages connected to the current evidence through either motif form the alternative pool after excluding retained evidence pages\. For both corpora, alternatives are deduplicated over the trajectory history: a document that has appeared in an earlier search observation is not shown again as an alternative in a later round\. Alternatives appear in search observations but are not opened\. Each search observation contains 10 candidates: all evidence documents assigned to that round are included, and alternatives fill the remaining slots\.

#### Synthetic supervision\.

We use Qwen3\.6\-35B\-A3B\-FP8\(Qwen Team,[2026](https://arxiv.org/html/2608.00432#bib.bib11)\)to infer proxy research objectives, generate search, open, and write rationales, and filter incidental Wikipedia links\. Generation uses a maximum of 32,768 tokens, temperature 0\.7, top\-pp0\.8, and top\-kk20\. The canonical medium response budget truncates rendered search observations to 400 tokens and open observations to 40,000 tokens\. The response\-length ablation uses search/open limits of 200/20,000 tokens for the short condition and 800/80,000 tokens for the long condition\.

#### Loss\-bearing tokens\.

System prompts, user objectives, and synthetic search/open observations are provided as context but masked from the language\-model loss\. Assistant reasoning, tool calls and their arguments, and final reports receive loss\. Under this objective, loss\-bearing tokens account for 8\.67% of DRP\-Paper and 7\.73% of DRP\-Web\.

Algorithm[A1](https://arxiv.org/html/2608.00432#alg1)summarizes the canonical interleaved construction\. The evidence resolver and alternative builder are instantiated with the citation\-graph and hyperlink\-graph rules described above\.

Algorithm A1Offline construction of a DRP trajectoryInput: Target passagerr; graph𝒢\\mathcal\{G\}; corpus𝒞\\mathcal\{C\}; corpus\-specific resolver𝖱𝖾𝗌𝗈𝗅𝗏𝖾\\mathsf\{Resolve\}and alternative builder𝖠𝗅𝗍\\mathsf\{Alt\}; response budgetsBS,BOB\_\{\\mathrm\{S\}\},B\_\{\\mathrm\{O\}\} Output: Predictive\-navigation trajectoryτ\\tauand token maskmm

1:

Er←𝖱𝖾𝗌𝗈𝗅𝗏𝖾​\(r,𝒢,𝒞\)E\_\{r\}\\leftarrow\\mathsf\{Resolve\}\(r,\\mathcal\{G\},\\mathcal\{C\}\)
2:

y←𝖭𝗈𝗋𝗆𝖺𝗅𝗂𝗓𝖾​\(r,Er\)y\\leftarrow\\mathsf\{Normalize\}\(r,E\_\{r\}\);

u←𝖨𝗇𝖿𝖾𝗋𝖮𝖻𝗃𝖾𝖼𝗍𝗂𝗏𝖾​\(r\)u\\leftarrow\\mathsf\{InferObjective\}\(r\)
3:

\(P1,…,PM\)←𝖯𝖺𝗋𝗍𝗂𝗍𝗂𝗈𝗇​\(Er,r\)\(P\_\{1\},\\ldots,P\_\{M\}\)\\leftarrow\\mathsf\{Partition\}\(E\_\{r\},r\)
4:

τ←\(u\)\\tau\\leftarrow\(u\);

h←\(u\)h\\leftarrow\(u\);

H←∅H\\leftarrow\\emptyset
5:for

i=1,…,Mi=1,\\ldots,Mdo

6:

Ni←𝖠𝗅𝗍​\(Pi,𝒢\)∖\(Er∪H\)N\_\{i\}\\leftarrow\\mathsf\{Alt\}\(P\_\{i\},\\mathcal\{G\}\)\\setminus\(E\_\{r\}\\cup H\)
7:

Ni←𝖲𝖾𝗅𝖾𝖼𝗍​\(Ni,10−\|Pi\|\)N\_\{i\}\\leftarrow\\mathsf\{Select\}\(N\_\{i\},10\-\|P\_\{i\}\|\)
8:

Ki←Pi∪NiK\_\{i\}\\leftarrow P\_\{i\}\\cup N\_\{i\}
9:

qi←𝖲𝗒𝗇𝗍𝗁𝖾𝗌𝗂𝗓𝖾𝖰𝗎𝖾𝗋𝗒​\(r,Pi,h\)q\_\{i\}\\leftarrow\\mathsf\{SynthesizeQuery\}\(r,P\_\{i\},h\)
10:

aiS←𝖲𝖾𝖺𝗋𝖼𝗁​\(qi\)a\_\{i\}^\{\\mathrm\{S\}\}\\leftarrow\\mathsf\{Search\}\(q\_\{i\}\);

ziS←𝖱𝖺𝗍𝗂𝗈𝗇𝖺𝗅𝖾​\(h,aiS\)z\_\{i\}^\{\\mathrm\{S\}\}\\leftarrow\\mathsf\{Rationale\}\(h,a\_\{i\}^\{\\mathrm\{S\}\}\)
11:

oiS←𝖱𝖾𝗇𝖽𝖾𝗋𝖲𝖾𝖺𝗋𝖼𝗁​\(Ki,𝒞;BS\)o\_\{i\}^\{\\mathrm\{S\}\}\\leftarrow\\mathsf\{RenderSearch\}\(K\_\{i\},\\mathcal\{C\};B\_\{\\mathrm\{S\}\}\)
12:Append

\(ziS,aiS,oiS\)\(z\_\{i\}^\{\\mathrm\{S\}\},a\_\{i\}^\{\\mathrm\{S\}\},o\_\{i\}^\{\\mathrm\{S\}\}\)to

τ\\tauand

hh;

H←H∪KiH\\leftarrow H\\cup K\_\{i\}
13:for

v∈Piv\\in P\_\{i\}do

14:

aO←𝖮𝗉𝖾𝗇​\(v\)a^\{\\mathrm\{O\}\}\\leftarrow\\mathsf\{Open\}\(v\);

zO←𝖱𝖺𝗍𝗂𝗈𝗇𝖺𝗅𝖾​\(h,aO\)z^\{\\mathrm\{O\}\}\\leftarrow\\mathsf\{Rationale\}\(h,a^\{\\mathrm\{O\}\}\)
15:

oO←𝖱𝖾𝗇𝖽𝖾𝗋𝖮𝗉𝖾𝗇​\(dv;BO\)o^\{\\mathrm\{O\}\}\\leftarrow\\mathsf\{RenderOpen\}\(d\_\{v\};B\_\{\\mathrm\{O\}\}\)
16:Append

\(zO,aO,oO\)\(z^\{\\mathrm\{O\}\},a^\{\\mathrm\{O\}\},o^\{\\mathrm\{O\}\}\)to

τ\\tauand

hh
17:endfor

18:endfor

19:Verify

𝖢𝗂𝗍𝖾​\(y\)=⋃i=1MPi\\mathsf\{Cite\}\(y\)=\\bigcup\_\{i=1\}^\{M\}P\_\{i\}
20:

zw←𝖶𝗋𝗂𝗍𝖾𝖱𝖺𝗍𝗂𝗈𝗇𝖺𝗅𝖾​\(h\)z\_\{\\mathrm\{w\}\}\\leftarrow\\mathsf\{WriteRationale\}\(h\); append

\(zw,𝖶𝗋𝗂𝗍𝖾,y\)\(z\_\{\\mathrm\{w\}\},\\mathsf\{Write\},y\)to

τ\\tau
21:Discard

τ\\tauif a rationale reveals hidden construction state or unrevealed evidence

22:

m←𝖠𝗀𝖾𝗇𝗍𝖳𝗈𝗄𝖾𝗇𝖬𝖺𝗌𝗄​\(τ\)m\\leftarrow\\mathsf\{AgentTokenMask\}\(\\tau\)
23:return

τ,m\\tau,m

### A\.2DRP\-Web Synthesis Prompts

The templates below document the LLM\-generated components of canonical interleaved DRP\-Web; they do not replace the construction defined in the main paper\. Starting from target reportrr, the constructor filters its outgoing\-link pages to recover evidence setErE\_\{r\}, partitions that evidence intoP1,…,PmP\_\{1\},\\ldots,P\_\{m\}, and augments each partition with graph\-derived candidate alternativesNiN\_\{i\}to form search candidate setKi=Pi∪NiK\_\{i\}=P\_\{i\}\\cup N\_\{i\}\. Search and open observations are then rendered offline fromKiK\_\{i\}and the source corpus\. The prompts infer proxy research objectiveuu, synthesize queriesqiq\_\{i\}, and generate search, open, and write rationales for the interleaved search–open–write trajectory\. Runtime fields below use descriptive angle\-bracket placeholders aligned with this notation\. Line breaks and dash glyphs are normalized for typesetting without changing the instructions\.

#### System prompts\.

Objective/query synthesis and write\-rationale generation use:

```
You are a deep research agent simulating how a researcher explores an
encyclopedia.
Output exactly in the requested format.
```

Search\- and open\-rationale generation use:

```
You are a deep research agent exploring an encyclopedia. You issue
search queries, read the pages you open, and reason about what to
look for next.
Output exactly in the requested format.
```

#### Objective, evidence, and query synthesis\.

For DRP\-Web, one call receives target reportrrand the outgoing\-link pages considered while resolvingErE\_\{r\}, represented by identifier, title, and lead text\. It filters incidental links, infers objectiveuu, partitions retained evidence into the per\-round setsPiP\_\{i\}, and synthesizes one queryqiq\_\{i\}per partition\. These outgoing\-link inputs are evidence\-resolution candidates, not the rendered search candidate setsKiK\_\{i\}; graph\-derived alternativesNiN\_\{i\}are added separately\.

```
You will write the report section below. It contains many hyperlinks,
but only SOME are genuine references whose content informs the
writing; others are incidental mentions (a person’s birthplace, a
generic term) not actually discussed.

SECTION:
---
<target_report_r>
---

CANDIDATE linked pages (id + title + lead):
<outgoing_link_pages>

Step 1 -- SELECT only the pages whose content is substantively
used/discussed in the section (drop incidental/navigational links).

Step 2 -- Plan a realistic multi-round search (typically 2-6 rounds)
that would naturally surface the SELECTED pages.
- GROUP the selected pages BY SUB-TOPIC:
  each round issues ONE query that retrieves a COHERENT GROUP of
  related pages at once -- not one page at a time.
- A round may target a single page ONLY if it is genuinely its own
  sub-topic. Do NOT split a natural group into many one-page rounds.
- Write each QUERY at the SUB-TOPIC / category level, not as the
  proper name of one specific page.
- Every SELECTED page assigned to exactly one round; dropped pages
  appear nowhere.
- QUERY: 3-7 keywords, no questions, operators, or quotes.

Output EXACTLY (no markdown):
USER_TOPIC: <proxy_objective_u>
DROPPED: <comma-separated wiki:ID you judged incidental, or NONE>
ROUND 1
QUERY: <query_q_i>
GOLD: <comma-separated selected wiki:ID in evidence partition P_i>
ROUND 2
...
```

Here, the prompt fieldGOLDdenotes evidence partitionPiP\_\{i\}\.

#### Search rationales\.

At search stepii, the template generates rationaleziSz\_\{i\}^\{\\mathrm\{S\}\}from objectiveuu, queryqiq\_\{i\}, and the causal historyhih\_\{i\}available at that position in the rendered trajectory\. The query targetqiq\_\{i\}is serialized afterhih\_\{i\}and its rationale, so its prediction during CPT conditions on the prior interaction history even though the constructor determines it offline\. The first\-round template is:

```
You are a researcher beginning to investigate this topic:

USER_TOPIC: <proxy_objective_u>

The first aspect you want to explore is captured by this search:
QUERY: <query_q_1>

Write your opening first-person thinking: what the topic is
fundamentally about, how you plan to build up an understanding of it,
and why starting with this aspect makes sense.

Hard rules:
- Speak ONLY about the subject matter. Do NOT mention searching,
  queries, results, or sources -- only the topic and your plan.
- Do NOT restate the query verbatim. Do NOT mention any instructions.

Output EXACTLY:
THINK: <2-4 sentences>
```

For subsequent rounds, the prompt includes the evidence opened in prior rounds:

```
You are a researcher building up a report on this topic:

USER_TOPIC: <proxy_objective_u>

Here is what you have LEARNED so far from your reading:
<rendered_history_h_i>

The aspect you want to explore NEXT is captured by this search:
QUERY: <query_q_i>

Write a SHORT first-person reflection that connects what you have
learned to this next step: what you now understand, and how exploring
this next aspect builds on or complements it. Frame it as forward
progress toward a fuller picture.

Hard rules:
- Speak ONLY about the subject matter and what you want to understand
  next.
- Do NOT mention searching, queries, results, pages, sources, or
  whether any earlier step "succeeded", "failed", "was
  relevant/irrelevant", or "lacked" anything. No commentary on the
  process -- only on the topic.
- Do NOT restate the query verbatim. Do NOT mention any instructions.

Output EXACTLY:
THINK: <2-4 sentences>
```

#### Open rationales\.

The open\-rationale template receives the rendered titles and snippets forKiK\_\{i\}, together with the titles of evidence documents inPiP\_\{i\}selected for opening:

```
You are a researcher building up a report on this topic:

USER_TOPIC: <proxy_objective_u>

What you have LEARNED so far:
<rendered_history_h_i>

You just looked over these candidate pages
(title -- brief description):
<search_candidates_K_i>

You have decided to READ IN FULL the following pages:
<selected_evidence_titles_P_i>

Write a SHORT first-person reflection explaining WHY these particular
pages are the ones worth reading in full for this topic -- what each
contributes and how it advances your understanding. Distinguish them
from the others only implicitly (by focusing on their relevance), not
by naming which were skipped.

Hard rules:
- Speak ONLY about the subject matter and why these pages matter to
  it.
- Do NOT mention searching, queries, results, ranking, or whether
  anything "failed", "was irrelevant", or "lacked" anything. No
  process commentary.
- Do NOT mention that pages were provided or listed. Do NOT mention
  instructions.

Output EXACTLY:
THINK: <2-4 sentences>
```

#### Write rationale\.

The final template generateszwz\_\{\\mathrm\{w\}\}from the complete rendered history before the model produces report targetyy:

```
You have finished gathering and reading sources for this topic:

USER_TOPIC: <proxy_objective_u>

You are about to write the report section.
In 2-3 sentences, think through how you will organize the writeup from
what you read -- what to lead with, how the pieces connect. Write
naturally as your own planning; do NOT mention instructions, lists,
or that any sources were provided.

Output EXACTLY:
THINK: <2-3 sentence writing plan>
```

#### Rationale filtering\.

Each rationale may be regenerated up to three times\. Consistent with the filtering rule in the main method, we retain it only if it does not reveal the hidden construction process or unrevealed evidence\. Deterministic checks reject references to supplied page lists or prompt instructions that would expose the offline oracle\.

### A\.3Continual Pretraining

Table[A1](https://arxiv.org/html/2608.00432#A1.T1)lists the continual\-pretraining configuration\. Qwen3\-14B\-Base is the primary backbone, and Qwen3\-30B\-A3B\-Base is used for the model\-scale analysis\. We pack examples without crossing the 128K\-token maximum sequence length\. Each 1B\-token run contains 500 optimizer steps at a 2M\-token global batch size and takes approximately six hours on 128 H20 96GB GPUs\. CPT uses OLMo\-core with training random seed 42\.

HyperparameterValueHyperparameterValueTraining tokens1BOptimizerAdamWMaximum sequence length128KAdam\(β1,β2\)\(\\beta\_\{1\},\\beta\_\{2\}\)\(0\.9,0\.999\)\(0\.9,0\.999\)Global batch size2M tokensAdamϵ\\epsilon10−810^\{\-8\}Peak learning rate4×10−54\\times 10^\{\-5\}Weight decay0Learning\-rate schedulecosine decay to 0Gradient clipping1\.0Warmup ratio0\.1PrecisionBF16Table A1:Continual\-pretraining hyperparameters\.
### A\.4Downstream Agent SFT

We use the 13,062 English GPT\-5\-generated trajectories released by DR Tulu\(Shaoet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib2)\)\.444[https://huggingface\.co/datasets/rl\-research/dr\-tulu\-sft\-data](https://huggingface.co/datasets/rl-research/dr-tulu-sft-data)Following its SFT recipe, models are fine\-tuned for five epochs on1/161/16,1/41/4, or all trajectories \(816, 3,265, and 13,062 examples, respectively\) using packed 16K\-token sequences, a 2M\-token global batch, BF16, and the optimizer settings in Table[A1](https://arxiv.org/html/2608.00432#A1.T1)\. The optimization configuration matches CPT except for the 16K maximum sequence length\. SFT also uses OLMo\-core, with training random seed fixed at 42 for every run\. Separately, for each low\-data fraction, we independently sample five subsets using data seeds 42, 2026, 5678, 31337, and 1234\. Within a data seed, every initialization receives exactly the same downstream trajectories\.

### A\.5Downstream Agentic RL

We initialize matched RL runs from the Base and DRP\-Web checkpoints trained on the same fixed1/161/16SFT subset\. Both follow the ARPO deep\-search setting\(Donget al\.,[2026](https://arxiv.org/html/2608.00432#bib.bib3)\)and are implemented with veRL using training random seed 42 and the GRPO advantage estimator\. We train for five epochs on the same 1K hard\-search examples with a global prompt batch of 64, a PPO mini\-batch of 64, and actor learning rate10−610^\{\-6\}\. Prompts and responses are limited to 2,000 and 6,192 tokens, respectively\. Synchronous vLLM tool rollouts use 12 samples per prompt, six initial rollouts, beam size 2, branch probability 0\.5, and entropy weight 0\.2\. Both the KL\-control and actor KL\-loss coefficients are zero\. We use tensor parallelism of 2, dynamic batching, gradient checkpointing, and validation before training; tool observations are excluded from the policy loss\. All RL data, rewards, rollouts, and optimization hyperparameters are identical between initializations\. We evaluate the post\-SFT checkpoint and checkpoints after 40 and 80 optimizer steps\. Each run takes approximately 24 hours on 64 H20 GPUs\.

### A\.6Agent Inference and Evaluation

Agents use the DR\-Tulu protocol with Serper for search and Jina for page extraction\. We permit at most 20 tool\-call rounds and 32K output tokens\. Decoding uses temperature 0\.6, top\-pp0\.95, and top\-kk20\.

The primary evaluation is the 50\-question English subset of DeepResearch Bench \(DRB\)\(Duet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib1)\)\. We evaluate DRB with its official GPT\-5\.5 RACE evaluator\.555[https://github\.com/Ayanami0730/deep˙research˙bench](https://github.com/Ayanami0730/deep_research_bench)We report RACE overall on its 0–100 scale as the primary score\. Score differences are absolute points on this scale, not relative percentage changes\. A missing final report receives zero\. Report rate is the percentage of questions with a valid final report, and tool calls are the mean number of search and open calls per question\.

Transfer evaluations use fixed sets of 500 examples from ResearchQA\(Li S\. Yifeiet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib15)\), WebWalkerQA\(Wuet al\.,[2025](https://arxiv.org/html/2608.00432#bib.bib16)\), and SimpleQA\(Weiet al\.,[2024](https://arxiv.org/html/2608.00432#bib.bib17)\)\. They use the same agent and a local Qwen3\-30B judge\. We report coverage for ResearchQA and accuracy for WebWalkerQA and SimpleQA\.

### A\.7ResearchQA Contamination Analysis

Because DRP\-Paper is derived from scholarly text, we test all 776 ResearchQA queries for verbatim overlap with the delivered DRP\-Paper corpus, comprising 78\.6K trajectories from 55,648 distinct arXiv source papers\. For each query, we check whether any of its wordkk\-grams appears in the DRP\-Paper training text\.

Table A2:Verbatim overlap between ResearchQA queries and DRP\-Paper training text\.No ResearchQA query shares even one 8\- or 13\-word span with the DRP\-Paper training corpus, providing no evidence of verbatim benchmark\-query leakage\.

## Appendix BStatistical Procedures

### B\.1Variation Across Sampled SFT Subsets

The five low\-data runs at each fraction vary the sampled SFT subset rather than the training random seed\. LetMa,sM\_\{a,s\}be the score of initializationaaafter SFT on the subset sampled with seedss, and letbbdenote the no\-CPT Base initialization\. For a matched comparison, we first compute

ds=Ma,s−Mb,s\.d\_\{s\}=M\_\{a,s\}\-M\_\{b,s\}\.\(A1\)We report the mean paired score gaind¯\\bar\{d\}and its 95% Student\-ttconfidence interval,

d¯±t0\.975,4​sd5,t0\.975,4=2\.776,\\bar\{d\}\\ \\mathbin\{\\pm\}\\ t\_\{0\.975,4\}\\frac\{s\_\{d\}\}\{\\sqrt\{5\}\},\\qquad t\_\{0\.975,4\}=2\.776,\(A2\)wheresds\_\{d\}is the sample standard deviation of the five paired differences\. This interval measures how the estimated gain changes when the downstream SFT subset changes\. It does not measure training\-seed variation or question\-level evaluation uncertainty\.

For the comparison between one\-quarter\-SFT DRP and full\-SFT Base, the Base score is a fixed reference\. We computeds=Ma,s1/4−Mbfulld\_\{s\}=M\_\{a,s\}^\{1/4\}\-M\_\{b\}^\{\\mathrm\{full\}\}for each of the five sampled one\-quarter subsets and apply the samettinterval\. Thus, the interval measures subset variation around the fixed full\-data reference\.

### B\.2Question\-Level Paired Bootstrap

For single\-checkpoint analyses, both compared systems answer the same DRB questions\. We jointly resample question IDs with replacement, preserving the pairing between systems, recompute the mean score difference for each bootstrap replicate, and report the 2\.5th and 97\.5th percentiles\. This paired bootstrap captures uncertainty over the evaluated questions while holding the trained checkpoints fixed\. It is therefore complementary to, rather than interchangeable with, the subset\-levelttintervals above\. We use 10,000 bootstrap replicates with random seed 42\.

### B\.3Repeated\-Evaluation Stability

To quantify variation from stochastic agent rollouts and evaluation, we independently evaluate the same DRP\-Web full\-SFT checkpoint three times under the same evaluation configuration\. Table[A3](https://arxiv.org/html/2608.00432#A2.T3)shows a mean score of 40\.64, a sample standard deviation of 0\.46, and a range of 0\.91 points\. This evaluation\-only spread is smaller than the matched low\-data gains in the main results, but it is measured for one full\-SFT checkpoint and does not replace the paired uncertainty analyses above\.

Table A3:Repeated evaluations of the same DRP\-Web full\-SFT checkpoint\. Scores use the 0–100 scale\.

## Appendix CAdditional Main Results

### C\.1Per\-Subset DeepResearch Bench Results

Tables[A4](https://arxiv.org/html/2608.00432#A3.T4)and[A5](https://arxiv.org/html/2608.00432#A3.T5)give the numerical results underlying the main\-paper sample\-efficiency figure\. The five columns at each low\-data fraction correspond to independently sampled, matched SFT subsets, not different training seeds\. All values are absolute score points on the 0–100 scale\.

Table A4:Per\-subset DeepResearch Bench results for the main Qwen3\-14B experiment\. Low\-data rows report five independently sampled SFT subsets and their mean and standard deviation\. Full\-SFT values are fixed single\-checkpoint references\.Table A5:Subset\-level paired score contrasts for the main experiment\. Intervals are 95% Student\-ttintervals over the five paired SFT subsets, as defined in Section[B](https://arxiv.org/html/2608.00432#A2)\. Full\-SFT Base is a fixed reference\.
### C\.2Full\-SFT Reference

The full\-SFT results come from single trained checkpoints and are therefore shown as fixed training references rather than as estimates over the five low\-data subsets\. The main claim concerning four\-times fewer trajectories is supported by comparing each one\-quarter\-SFT DRP run with the same full\-SFT Base checkpoint and its designated evaluation\. We keep this distinction explicit to avoid interpreting the displayed interval as training\-seed uncertainty for full SFT\. Repeated evaluation of the DRP\-Web full\-SFT checkpoint is reported separately in Table[A3](https://arxiv.org/html/2608.00432#A2.T3)\.

### C\.3RACE Dimension Breakdown

Table[A6](https://arxiv.org/html/2608.00432#A3.T6)gives the numerical values underlying the RACE dimension figure in the main paper\. Both DRP variants improve comprehensiveness, insight, instruction following, and readability at1/161/16and1/41/4SFT\. This rules out an explanation in which the overall gain is caused solely by style or report validity\.

Table A6:Numerical DRB RACE\-dimension results\. Values at1/161/16and1/41/4SFT are means over five independently sampled, matched SFT subsets; full\-SFT values are fixed single\-run references\. All scores use the 0–100 scale\.
### C\.4General\-Capability Scores

The general\-capability suite comprises ARC, OpenBookQA, HellaSwag, PIQA, GSM8K, CommonsenseQA, MMLU\-Redux, DROP, HotpotQA, TriviaQA, WikiTableQuestions, 2WikiMultiHopQA, SQuAD 2\.0, and MuSiQue\(Clarket al\.,[2018](https://arxiv.org/html/2608.00432#bib.bib26); Mihaylovet al\.,[2018](https://arxiv.org/html/2608.00432#bib.bib27); Zellerset al\.,[2019](https://arxiv.org/html/2608.00432#bib.bib28); Bisket al\.,[2020](https://arxiv.org/html/2608.00432#bib.bib29); Cobbeet al\.,[2021](https://arxiv.org/html/2608.00432#bib.bib30); Talmoret al\.,[2019](https://arxiv.org/html/2608.00432#bib.bib31); Gema and others,[2025](https://arxiv.org/html/2608.00432#bib.bib32); Duaet al\.,[2019](https://arxiv.org/html/2608.00432#bib.bib33); Yanget al\.,[2018](https://arxiv.org/html/2608.00432#bib.bib34); Joshiet al\.,[2017](https://arxiv.org/html/2608.00432#bib.bib35); Pasupat and Liang,[2015](https://arxiv.org/html/2608.00432#bib.bib36); Hoet al\.,[2020](https://arxiv.org/html/2608.00432#bib.bib37); Rajpurkaret al\.,[2018](https://arxiv.org/html/2608.00432#bib.bib38); Trivediet al\.,[2022](https://arxiv.org/html/2608.00432#bib.bib39)\)\. Figure[A1](https://arxiv.org/html/2608.00432#A3.F1)visualizes the per\-benchmark results, and Table[A7](https://arxiv.org/html/2608.00432#A3.T7)gives their numerical values\. Replacing half of the 2B\-token general CPT budget with DRP\-Web changes the 14\-benchmark average from 73\.1 to 73\.4, while improving DeepResearch Bench after matched1/161/16trajectory SFT from 33\.1 to 36\.8\.

![Refer to caption](https://arxiv.org/html/2608.00432v1/Figures/fig_general_capability.png)Figure A1:Per\-benchmark general\-capability results and downstream DeepResearch Bench performance under a fixed 2B\-token Qwen3\-14B CPT budget\. The control uses 2B general midtraining tokens; the mixed model uses 1B general and 1B DRP\-Web tokens\. General benchmarks are evaluated after CPT, and DeepResearch Bench after matched1/161/16DR\-Tulu SFT\.Table A7:General\-capability preservation and downstream utility under a fixed 2B\-token Qwen3\-14B CPT budget\. General uses 2B general tokens; Mixed uses 1B general plus 1B DRP\-Web tokens\. General benchmarks evaluate post\-CPT checkpoints; DRB evaluates both after matched1/161/16DR\-Tulu SFT\. Bold marks the higher score for each benchmark; both entries are bold for ties\.

## Appendix DSynthetic\-Data Cost Accounting

We estimate the cost of constructing training data, separately from the cost of training on that data\. The comparison uses public list prices as a common reference rather than our historical expenditure: $10 per million output tokens for GPT\-5666[https://developers\.openai\.com/api/docs/models/gpt\-5](https://developers.openai.com/api/docs/models/gpt-5)and $1 per million output tokens for Qwen3\.6\-35B\-A3B on OpenRouter\.777[https://openrouter\.ai/qwen/qwen3\.6\-35b\-a3b](https://openrouter.ai/qwen/qwen3.6-35b-a3b)All token volumes are serialized\-data estimates computed as characters divided by 4\.5\.

For a conservative, reproducible comparison, we price only visible output text\. Input\-token billing and external search/retrieval charges are excluded because historical request logs are unavailable\. This choice favors expert SFT: an online GPT\-5 trajectory repeatedly consumes the growing interaction history as input and invokes real search/open services, whereas DRP observations are rendered directly from the evidence graph and source corpus\. We nevertheless count*all*80\.42M non\-observation tokens in the DRP\-Web 1B slice at the Qwen output price\. This is an upper bound because the category includes source\-derived target reports in addition to generated proxy objectives, rationales, and tool arguments\.

Table A8:Standardized data\-construction cost\. Full SFT contains an estimated 34\.76M visible GPT\-5 output tokens\. The DRP recipe combines 8\.71M GPT\-5 output tokens \($87\.1\) with an upper bound of 80\.42M Qwen output\-equivalent tokens \(at most $80\.4\)\. Thus, it reduces the output\-price proxy by at least 51\.8% while using 75% fewer expert trajectories and obtaining a higher DeepResearch Bench score\. Training costs are outside this accounting boundary\.![Refer to caption](https://arxiv.org/html/2608.00432v1/x8.png)Figure A2:The canonical one\-quarter SFT subset uses approximately 75% fewer expert trajectories, real environment calls, visible teacher\-output tokens, and observation tokens than full SFT\. DRP\-Web constructs its search/open observations offline and therefore adds no real environment calls\.The resulting comparison isolates the resource that DRP is designed to save: expensive downstream expert supervision\. Base \+ Full SFT costs approximately $347\.6 under the output\-price proxy\. DRP\-Web synthesis plus one\-quarter SFT costs at most $167\.5, a reduction of at least 51\.8%, while improving the score from 38\.5 to 39\.6\. It also reduces real environment calls from 47,147 to 11,597 \(75\.4%\)\. The advantage is therefore not that pretraining is free, but that an inexpensive offline generator and naturally occurring evidence graphs can amortize agent supervision before downstream post\-training\. This is especially relevant when task\-specific expert trajectories are scarce or when online environment feedback is slow\.

## Appendix EConstruction and Ablation Details

### E\.1Raw\-Domain Controls

Raw\-Web and Raw\-Paper use the same source\-document pools as DRP\-Web and DRP\-Paper, respectively\. They match the 1B\-token CPT budget but remove proxy queries, reasoning traces, tool calls, and graph\-derived relevance relations\. Consequently, the DRP–Raw comparison isolates the effect of restructuring the same domains into predictive\-navigation supervision\.

### E\.2Evidence\-Mismatched DRP\-Web

Evidence\-Mismatched DRP\-Web preserves the proxy objectives, generated reasoning, action schedule, protocol rendering, response\-length distribution, and training budget of canonical DRP\-Web\. We replace the graph\-derived search and open observations with random in\-domain evidence\. The ablation therefore breaks the alignment among query, observation, and subsequent action while preserving the agent\-like surface form\.

### E\.3Pseudo\-operation Topologies

We useS,O, andWfor search, open, and write\. The search\-only condition omits full\-document open observations\. The batchedSSSOWand interleavedSOSOWnames denote topology rather than fixed\-length traces\. The former performs a variable number of searches before opening selected evidence and writing; the latter performs a variable number of search–open rounds before writing\. Interleaving allows later synthetic rationales and queries to condition on previously opened evidence\.

### E\.4Protocol Rendering

The DR\-Tulu and Hermes conditions contain the same objectives, think traces, tool calls, observations, and final reports\. They differ only in serialization\. DR\-Tulu groups the constructed interaction in the compact downstream protocol, whereas Hermes alternates assistant tool\-call turns and tool\-response turns\. This control tests protocol alignment rather than a change in supervision\.

### E\.5Response Length and Loss Masking

The response\-length ablation changes the amount of evidence rendered in masked tool observations\. Shorter responses reduce training cost and increase the fraction of loss\-bearing tokens, but may omit information required for the next decision or final report\. Separately, the no\-mask condition computes loss on constructed tool observations in addition to assistant outputs; all other data and optimization settings remain fixed\.

## Appendix FAdditional Limitations

The current experiments use English downstream trajectories and one released trajectory source\. The primary DRB evaluation has 50 English questions, making paired evaluation and transparent uncertainty reporting especially important\. The five\-run design measures downstream subset sensitivity but does not estimate the full interaction among subset sampling, training randomness, decoding randomness, and judge variation\. Most construction ablations use one fixed SFT subset and paired question\-level bootstrap intervals\. Finally, DRP’s gain attenuates with more downstream SFT data\.

Similar Articles

Mind DeepResearch Technical Report

Hugging Face Daily Papers

MindDR is a multi-agent deep research framework using a three-agent architecture (Planning, DeepSearch, Report) and a four-stage training pipeline, achieving competitive performance with ~30B-parameter models on multiple benchmarks. Developed by Li Auto and deployed as an online product, it also introduces MindDR Bench, a 500-query Chinese benchmark for evaluating deep research capabilities.