Search-G1: Grounded Search Agents via Representation-Based Intrinsic Rewards

# Search-G1: Grounded Search Agents via Representation-Based Intrinsic Rewards
Source: [https://arxiv.org/html/2608.07531](https://arxiv.org/html/2608.07531)
Ruoxi Cheng1,2, Haoxuan Ma3, Hongyi Zhang4, Junming Zhang5, Ranjie Duan2, Qiaolin Xia2, Hao Wang2,†, Yu Lu2, Haibo Shi2, Xingjun Ma1,† 1Fudan University2Tencent3Nanjing University 4Nanyang Technological University5Shanghai Jiao Tong University

###### Abstract

Search\-augmented language agents should retrieve external information only when necessary and ground their answers in retrieved evidence\. Existing external rewards provide either sparse outcome supervision or richer feedback from process annotations and LLM judges\. Outcome rewards scale readily but cannot distinguish grounded retrieval from redundant search, whereas richer signals require costly annotation or inference during training\. Internal rewards based on policy\-side signals such as entropy, likelihood, or information gain are graded and inexpensive to evaluate, yet mainly reflect model confidence rather than evidence grounding\. We proposeSearch\-G1, a representation\-based intrinsic reward framework that measures the operational grounding of an agent’s answers through two intervention\-calibrated readouts\. A prompt\-state readout predicts closed\-book sufficiency, whose complement defines policy\-relativeretrieval necessity; an answer\-commit readout estimatesevidence reliancefrom answer\-stage sensitivity to evidence deletion\. Together, they provide additional credit to correct searched trajectories when retrieval is estimated necessary and the answer is evidence\-sensitive, favor correct direct answers when closed\-book knowledge suffices, and penalize repeated search\. After calibration, reward scoring requires neither process annotations nor LLM\-as\-judge inference during policy optimization\. Because reinforcement learning changes policy representations, Search\-G1 periodically refits both readouts on trajectories from the latest checkpoint, allowing the reward to co\-evolve with the policy\. Experiments across multiple search\-based question\-answering benchmarks and two model scales show that Search\-G1 improves the grounding–search\-cost trade\-off, producing shorter response\-side trajectories at competitive task accuracy\. Code is available at[Rosy0912/Search\-G1](https://github.com/Rosy0912/Search-G1)\.

22footnotetext:Corresponding to[vichwang@tencent\.com](https://arxiv.org/html/2608.07531v1/mailto:[email protected]);[xingjunma@fudan\.edu\.cn](https://arxiv.org/html/2608.07531v1/mailto:[email protected])\. This work was conducted during Ruoxi Cheng’s participation in the Tencent \(Yuanbao AI Search\) Rhino\-Bird Research Elite Program \(Industrial Supervisor: Hao Wang; Academic Supervisor: Xingjun Ma\)\.## 1Introduction

Search agents augment large language models \(LLMs\) with an external retrieval loop\. They issue queries, inspect documents, reason over evidence, and then commit an answer\[[32](https://arxiv.org/html/2608.07531#bib.bib1),[46](https://arxiv.org/html/2608.07531#bib.bib2)\]\. Recent Search\-R1\-style systems show that reinforcement learning can induce this behavior\[[19](https://arxiv.org/html/2608.07531#bib.bib3),[14](https://arxiv.org/html/2608.07531#bib.bib44),[3](https://arxiv.org/html/2608.07531#bib.bib46)\]\. Yet the interface only determines what an agent*can*do\. The reward determines which behavior policy optimization reinforces\. A capable agent should perform*selective grounded search*: retrieve when needed, and ground the answer in what was retrieved\. This requires two distinct judgments\.*Retrieval necessity*\[[8](https://arxiv.org/html/2608.07531#bib.bib10)\]asks whether the current policy can answer without retrieval\.*Evidence reliance*asks whether a searched answer actually depends on the retrieved context\.

![Refer to caption](https://arxiv.org/html/2608.07531v1/intro.png)Figure 1:Reward signals for agentic search RL\. External rewards range from sparse outcomes to process supervision, tool feedback, and judge signals\. Internal rewards provide graded signals but often capture confidence rather than evidence use\. Search\-G1 calibrates two policy readouts through interventions: a prompt readout estimates whether the policy can answer without retrieval, and an answer readout estimates reliance on retrieved evidence\.The need for retrieval varies with both question structure and policy knowledge\. On the held\-out NQ split\[[22](https://arxiv.org/html/2608.07531#bib.bib32)\], a base Qwen2\.5\-3B\[[37](https://arxiv.org/html/2608.07531#bib.bib31)\]answers14%14\\%of questions correctly without retrieval\. This identifies a nontrivial subset on which search may be redundant, while leaving a much larger subset that may benefit from external evidence\. Evidence\-heavy benchmarks pose a different challenge: multi\-step aggregation\. In HotpotQA\[[45](https://arxiv.org/html/2608.07531#bib.bib33)\],79\.9%79\.9\\%of questions are bridge type; MuSiQue\[[38](https://arxiv.org/html/2608.07531#bib.bib38)\]consists entirely of22–44\-hop questions \([Table˜A3](https://arxiv.org/html/2608.07531#S7.T3)\)\. However, these structural annotations characterize the task, not how much retrieval a particular policy needs\[[17](https://arxiv.org/html/2608.07531#bib.bib37),[38](https://arxiv.org/html/2608.07531#bib.bib38)\]\. Outcome\-only or necessity\-agnostic rewards do not explicitly distinguish redundant search from retrieval that supplies missing evidence\. We therefore make both judgments policy\-relative: necessity is derived from closed\-book sufficiency, while reliance is measured through answer\-stage sensitivity to evidence deletion—an operational proxy rather than a causal certificate\.

Existing reward designs fall into two broad families, as shown in[Figure˜1](https://arxiv.org/html/2608.07531#S1.F1)\.External rewardsinclude sparse answer correctness\[[34](https://arxiv.org/html/2608.07531#bib.bib15),[19](https://arxiv.org/html/2608.07531#bib.bib3)\], process reward models\[[42](https://arxiv.org/html/2608.07531#bib.bib4),[44](https://arxiv.org/html/2608.07531#bib.bib5)\], tool\-use supervision\[[24](https://arxiv.org/html/2608.07531#bib.bib16)\], and LLM\-as\-judge feedback\[[18](https://arxiv.org/html/2608.07531#bib.bib17)\]\. Outcome rewards scale naturally\. However, they assign the same score to an answer grounded in retrieved evidence and one recalled from parametric memory after redundant search\. Richer external signals inspect trajectories more directly, but require annotations or additional model calls and may be brittle on multi\-step traces\[[25](https://arxiv.org/html/2608.07531#bib.bib6)\]\.Internal rewardsinstead derive graded signals from the policy itself\[[6](https://arxiv.org/html/2608.07531#bib.bib18)\], such as entropy\[[5](https://arxiv.org/html/2608.07531#bib.bib13)\], answer likelihood or information gain\[[41](https://arxiv.org/html/2608.07531#bib.bib14)\], and logit\-level confidence\. They avoid external supervision at reward time\. Yet they primarily characterize uncertainty, not evidence use: a model may remain confident because it already knows an answer, even after issuing a search\[[27](https://arxiv.org/html/2608.07531#bib.bib19),[40](https://arxiv.org/html/2608.07531#bib.bib20),[36](https://arxiv.org/html/2608.07531#bib.bib8)\]\. The missing capability is not another confidence signal\. It is a low\-cost reward that jointly measures policy\-relative retrieval necessity and answer\-level evidence reliance\.

To address these limitations, we introduceSearch\-G1, a representation\-based intrinsic reward that scores how well an agent grounds its answers in retrieved evidence, built on two intervention\-calibrated readouts\. The first operates on the prompt state and predicts closed\-book sufficiency; its complement gives retrieval necessity\. The second operates at answer commitment and estimates evidence reliance from sensitivity to evidence deletion\. Counterfactual interventions provide the calibration targets\. After calibration, the readouts amortize these tests during rollout scoring, without process annotations or external LLM\-judge calls\. Here,*intrinsic*refers to reward\-time scoring from the policy’s own representations, not to label\-free calibration\. We combine the two scores in a correctness\-first, normalization\-aware trajectory reward\. Correct searched trajectories receive necessity\-conditioned reliance credit\. Correct direct trajectories receive closed\-book\-sufficiency credit, while repeated search incurs a bounded cost\. This shaping can restore nonzero reward variation in otherwise tied all\-correct groups, for which outcome\-only GRPO yields zero advantage\. However, the underlying measurements cannot remain static\. Reinforcement learning changes the representations on which both readouts depend\. Search\-G1 therefore periodically refits them on trajectories from the latest policy checkpoint, closing a measurement–credit–adaptation loop\. More broadly, this treats search\-agent training as adaptive measurement: as the policy’s knowledge boundary shifts, so should the reward used to assess evidence use\. Across benchmarks and model scales, Search\-G1 achieves a favorable grounding–search\-use trade\-off while maintaining competitive task accuracy\.

Our main contributions are as follows:

- •We proposeSearch\-G1, a representation\-based intrinsic reward with two intervention\-calibrated readouts\. Together, they estimate policy\-relative retrieval necessity and operational evidence reliance\.
- •We design a normalization\-aware credit assignment scheme that can restore reward variation in otherwise tied all\-correct groups\. A periodic refitting mechanism keeps both readouts aligned with the evolving policy\.
- •Across four QA benchmarks and two model scales, Search\-G1 improves operational grounding and shortens trajectories while keeping accuracy competitive, with supporting diagnostics on estimator fidelity, reward tie\-breaking, and representation drift\.

## 2Related Work

### 2\.1Search\-Agent Rewards and Credit Assignment

Search\-R1\-style GRPO methods show that outcome rewards can induce search\[[19](https://arxiv.org/html/2608.07531#bib.bib3),[11](https://arxiv.org/html/2608.07531#bib.bib23)\], but exact\-match feedback is sparse and grounding\-blind: it cannot distinguish answers produced*because of search*from answers recalled from memory*after search*\. External feedback provides finer credit through process rewards\[[44](https://arxiv.org/html/2608.07531#bib.bib5)\], judge models\[[26](https://arxiv.org/html/2608.07531#bib.bib43)\], and counterfactual or influence\-based attribution\[[13](https://arxiv.org/html/2608.07531#bib.bib22),[28](https://arxiv.org/html/2608.07531#bib.bib52)\], but requires additional annotation, verification, or inference\. Low\-cost policy\-internal signals instead use state\-anchored groups\[[7](https://arxiv.org/html/2608.07531#bib.bib42)\], entropy\[[5](https://arxiv.org/html/2608.07531#bib.bib13)\], answer likelihood or information gain\[[41](https://arxiv.org/html/2608.07531#bib.bib14)\], and self\-distilled references\[[23](https://arxiv.org/html/2608.07531#bib.bib48)\]; they provide denser feedback but mainly sharpen the optimization signal, reflecting model confidence or progress rather than whether an answer actually uses retrieved evidence\. Grounding\-oriented methods more directly target evidence use with an external judge for answer–support alignment\[[26](https://arxiv.org/html/2608.07531#bib.bib43)\], post\-hoc information\-credit estimation\[[28](https://arxiv.org/html/2608.07531#bib.bib52)\], or model\-internal saliency attribution\[[31](https://arxiv.org/html/2608.07531#bib.bib55)\]\. Knowledge\- or search\-boundary methods further regulate retrieval, stopping, and abstention through rollout contrasts, certainty–quality partitions, or boundary\-informed distillation\[[8](https://arxiv.org/html/2608.07531#bib.bib10),[43](https://arxiv.org/html/2608.07531#bib.bib24)\], but do not directly credit answer\-level dependence on retrieved evidence\[[36](https://arxiv.org/html/2608.07531#bib.bib8),[9](https://arxiv.org/html/2608.07531#bib.bib9)\]\. Search\-G1 instead periodically refits intervention\-calibrated hidden\-state readouts on the latest policy snapshot and combines their necessity and reliance scores in a correctness\-first trajectory reward\.

### 2\.2Internal Signals of Evidence Dependence

Evidence dependence requires separating parametric recall from contextual use\. Probing exposes relational facts\[[30](https://arxiv.org/html/2608.07531#bib.bib25)\], feed\-forward layers behave as key–value memories\[[12](https://arxiv.org/html/2608.07531#bib.bib26)\], and representation analyses trace or edit parametric and contextual knowledge\[[16](https://arxiv.org/html/2608.07531#bib.bib35),[49](https://arxiv.org/html/2608.07531#bib.bib34)\]; complementary pre\-training externalizes factual values to explicit memory\[[50](https://arxiv.org/html/2608.07531#bib.bib36)\]\. Models can self\-assess correctness\[[20](https://arxiv.org/html/2608.07531#bib.bib27)\], while hidden states encode truthfulness, entity knowledge, and tool\-selection errors\[[1](https://arxiv.org/html/2608.07531#bib.bib28),[10](https://arxiv.org/html/2608.07531#bib.bib54),[15](https://arxiv.org/html/2608.07531#bib.bib12)\]\. Such signals support RAG hallucination detection\[[35](https://arxiv.org/html/2608.07531#bib.bib53)\], factuality\-oriented decoding\[[4](https://arxiv.org/html/2608.07531#bib.bib29)\], activation\-space control\[[2](https://arxiv.org/html/2608.07531#bib.bib49),[29](https://arxiv.org/html/2608.07531#bib.bib50)\], and diagnosis of tool\-augmented traces\[[25](https://arxiv.org/html/2608.07531#bib.bib6),[48](https://arxiv.org/html/2608.07531#bib.bib30)\]\. Most, however, target fixed\-model diagnosis or inference\-time control rather than RL credit\. Agent RL has instead used retrospective critics for dense search feedback\[[47](https://arxiv.org/html/2608.07531#bib.bib51)\], hidden\-state diversity for exploration\[[39](https://arxiv.org/html/2608.07531#bib.bib11)\], and fixed offline heads over hidden\-state and attention features for step\-level rewards\[[21](https://arxiv.org/html/2608.07531#bib.bib7)\]\. In contrast, Search\-G1 credits answer\-level evidence reliance together with retrieval necessity rather than step\-level correctness, and periodically refits its policy\-conditioned readouts so the reward co\-evolves with the policy instead of relying on a frozen snapshot\.

![Refer to caption](https://arxiv.org/html/2608.07531v1/pipeline.png)Figure 2:Pipeline ofSearch\-G1\. We first use a frozen policy snapshot and counterfactual interventions to calibrate two readouts: a prompt\-state readout estimates closed\-book sufficiency, whose complement gives retrieval necessity, and an answer\-commit readout estimates answer sensitivity to evidence removal\. We then combine their scores with correctness and repeated\-search cost to form a correctness\-first trajectory reward\. Finally, we update the policy using group\-relative advantages over policy\-generated tokens and periodically refit the readouts on the latest checkpoint, closing the measurement–credit–adaptation loop\.

## 3Methodology

In this section, we presentSearch\-G1, a reinforcement\-learning framework for selective grounded search\. As illustrated in[Figure˜2](https://arxiv.org/html/2608.07531#S2.F2), we first use policy\-relative counterfactuals to calibrate two lightweight state readouts: one estimates whether the policy can answer without retrieval, and the other estimates whether its answer changes when retrieved evidence is removed\. We then combine these estimates with answer correctness and repeated\-search cost to construct a trajectory reward that credits evidence\-sensitive search when retrieval is necessary\. Finally, we update the policy using group\-relative advantages over policy\-generated tokens and periodically refit the readouts on the latest checkpoint, closing the measurement–credit–adaptation loop\.

### 3\.1Problem Formulation

Given a questionqiq\_\{i\}, gold answeryi⋆y\_\{i\}^\{\\star\}, and search\-enabled promptPiP\_\{i\}, the rollout policy samples a group ofG≥2G\\geq 2trajectories,

τi,j∼iidπold\(⋅∣Pi\),j=1,…,G\.\\tau\_\{i,j\}\\overset\{\\mathrm\{iid\}\}\{\\sim\}\\pi\_\{\\mathrm\{old\}\}\(\\cdot\\mid P\_\{i\}\),\\qquad j=1,\\ldots,G\.The rollout policyπold\\pi\_\{\\mathrm\{old\}\}remains fixed whileπθ\\pi\_\{\\theta\}is updated on the sampled group\. Each trajectory contains reasoning, search actions, retrieved observations, and a final answera^i,j\\hat\{a\}\_\{i,j\}\. Letvi,j∈\{0,1\}v\_\{i,j\}\\in\\\{0,1\\\}indicate validity under the output parser, and letei,j=V​\(a^i,j,yi⋆\)∈\{0,1\}e\_\{i,j\}=V\(\\hat\{a\}\_\{i,j\},y\_\{i\}^\{\\star\}\)\\in\\\{0,1\\\}indicate answer correctness, withei,j=0e\_\{i,j\}=0for invalid outputs\. The statisticNi,j∈ℤ≥0N\_\{i,j\}\\in\\mathbb\{Z\}\_\{\\geq 0\}counts parsed search actions:Ni,j=0N\_\{i,j\}=0denotes a direct trajectory andNi,j\>0N\_\{i,j\}\>0a searched one\.

The modeling problem is to convert two latent, policy\-relative judgments—whether retrieval was needed forqiq\_\{i\}and whether a searched answer is sensitive to removal of the retrieved evidence—into trajectory\-level credit\. We seek a rewardri,jr\_\{i,j\}that preserves correctness\-first optimization while favoring evidence\-sensitive search when retrieval is necessary, direct answering when the policy is sufficient closed\-book, and discouraging repeated search among correct searched trajectories\. Because GRPO normalizes rewards within each rollout group, this credit must also induce meaningful within\-group variation\.

### 3\.2Representation Calibration

At calibration roundmm, we freeze the latest policy checkpointθ¯m\\bar\{\\theta\}\_\{m\}\. The snapshot generates counterfactual completions and encodes the hidden states used by two readouts: the prompt statehp,ih\_\{p,i\}at the end of the initial prompt and the answer\-commit statehans,i,jh\_\{\\mathrm\{ans\},i,j\}at the final answer token\.

##### Evidence Reliance\.

For a valid searched trajectory, letCi,jC\_\{i,j\}be its realized context immediately before the answer\. The interventionT​\(Ci,j\)T\(C\_\{i,j\}\)removes retrieved observations while retaining the question, search history, and generated reasoning\. We define the answer\-change target and its readout as

si,jchg\\displaystyle s\_\{i,j\}^\{\\mathrm\{chg\}\}=𝟏​\[ν​\(a^i,j\)≠ν​\(Aθ¯mdet​\(T​\(Ci,j\)\)\)\],\\displaystyle=\\mathbf\{1\}\\\!\\left\[\\nu\(\\hat\{a\}\_\{i,j\}\)\\neq\\nu\\\!\\left\(A\_\{\\bar\{\\theta\}\_\{m\}\}^\{\\mathrm\{det\}\}\(T\(C\_\{i,j\}\)\)\\right\)\\right\],\(1\)di,j\\displaystyle d\_\{i,j\}=Dϕ\(m\)​\(hans,i,j\)∈\[0,1\],\\displaystyle=D\_\{\\phi^\{\(m\)\}\}\(h\_\{\\mathrm\{ans\},i,j\}\)\\in\[0,1\],whereν\\nunormalizes answer strings\. Thusdi,jd\_\{i,j\}estimates answer\-stage sensitivity to evidence deletion, giving an operational reliance signal for reward shaping\.

##### Retrieval Necessity\.

From a fixed retrieval\-disabled promptPicbP\_\{i\}^\{\\mathrm\{cb\}\}, we obtain a closed\-book target and predict it from the actual search\-enabled prompt state:

zi\\displaystyle z\_\{i\}=Vcb​\(Aθ¯mdet​\(Picb\),yi⋆\)∈\{0,1\},\\displaystyle=V\_\{\\mathrm\{cb\}\}\\\!\\left\(A\_\{\\bar\{\\theta\}\_\{m\}\}^\{\\mathrm\{det\}\}\(P\_\{i\}^\{\\mathrm\{cb\}\}\),y\_\{i\}^\{\\star\}\\right\)\\in\\\{0,1\\\},\(2\)bi\\displaystyle b\_\{i\}=Bψ\(m\)​\(hp,i\)∈\[0,1\],ni=1−bi\.\\displaystyle=B\_\{\\psi^\{\(m\)\}\}\(h\_\{p,i\}\)\\in\[0,1\],\\qquad n\_\{i\}=1\-b\_\{i\}\.Herebib\_\{i\}estimates closed\-book sufficiency under the specified policy and prompt protocol, andnin\_\{i\}measures relative retrieval necessity\. The target is generated fromPicbP\_\{i\}^\{\\mathrm\{cb\}\}, whereas the readout operates on the pre\-search state induced byPiP\_\{i\}, making the score available before the agent acts\.VcbV\_\{\\mathrm\{cb\}\}uses normalized answer inclusion; its agreement with the task EM verifierVVis audited in[Section˜7\.3\.4](https://arxiv.org/html/2608.07531#S7.SS3.SSS4)\.

Let𝒟D\(m\)\\mathcal\{D\}\_\{D\}^\{\(m\)\}contain valid searched trajectories and𝒟B\(m\)\\mathcal\{D\}\_\{B\}^\{\(m\)\}contain question\-level closed\-book examples\. The two readouts are fitted with

ℒcal=𝔼𝒟D\(m\)​\[BCE​\(di,j,si,jchg\)\]\+𝔼𝒟B\(m\)​\[BCE​\(bi,zi\)\]\.\\mathcal\{L\}\_\{\\mathrm\{cal\}\}=\\mathbb\{E\}\_\{\\mathcal\{D\}\_\{D\}^\{\(m\)\}\}\\\!\\left\[\\mathrm\{BCE\}\(d\_\{i,j\},s\_\{i,j\}^\{\\mathrm\{chg\}\}\)\\right\]\+\\mathbb\{E\}\_\{\\mathcal\{D\}\_\{B\}^\{\(m\)\}\}\\\!\\left\[\\mathrm\{BCE\}\(b\_\{i\},z\_\{i\}\)\\right\]\.\(3\)This expression gives the shared calibration criterion; candidate\-specific class weighting, regularization, architecture selection, and label\-validity audits are detailed in[Sections˜7\.2\.1](https://arxiv.org/html/2608.07531#S7.SS2.SSS1),[7\.3\.1](https://arxiv.org/html/2608.07531#S7.SS3.SSS1)and[7\.3\.4](https://arxiv.org/html/2608.07531#S7.SS3.SSS4)\.

### 3\.3Necessity\-Gated Reward

For a searched trajectory, Search\-G1 combines evidence reliance and retrieval necessity into a necessity\-gated reliance score

gi,j=di,j​ni\.g\_\{i,j\}=d\_\{i,j\}n\_\{i\}\.\(4\)This multiplicative gate is large only when the answer is evidence\-sensitive and closed\-book sufficiency is low\. Correct direct trajectories instead usebib\_\{i\}\.

Within the correct\-searched reward branch, the first search is unpenalized, while any trajectory with two or more search actions activates a unit cost:

ci,j=𝟏​\[Ni,j\>1\]\.c\_\{i,j\}=\\mathbf\{1\}\[N\_\{i,j\}\>1\]\.\(5\)Thus direct and single\-search trajectories have zero cost, whereas repeated search has unit cost\.

Letrinvr\_\{\\mathrm\{inv\}\}andrwrongr\_\{\\mathrm\{wrong\}\}denote the rewards for invalid and wrong outputs\. The Search\-G1 reward is

ri,j=\{rinv,vi,j=0,rwrong,vi,j=1,ei,j=0,1\+λg​gi,j−η​ci,j,vi,j=1,ei,j=1,Ni,j\>0,1\+αcb​bi,vi,j=1,ei,j=1,Ni,j=0\.r\_\{i,j\}=\\begin\{cases\}r\_\{\\mathrm\{inv\}\},&v\_\{i,j\}=0,\\\\\[2\.0pt\] r\_\{\\mathrm\{wrong\}\},&v\_\{i,j\}=1,\\ e\_\{i,j\}=0,\\\\\[2\.0pt\] 1\+\\lambda\_\{g\}g\_\{i,j\}\-\\eta c\_\{i,j\},&v\_\{i,j\}=1,\\ e\_\{i,j\}=1,\\ N\_\{i,j\}\>0,\\\\\[2\.0pt\] 1\+\\alpha\_\{\\mathrm\{cb\}\}b\_\{i\},&v\_\{i,j\}=1,\\ e\_\{i,j\}=1,\\ N\_\{i,j\}=0\.\\end\{cases\}\(6\)
For a correct searched trajectory and a correct direct trajectory from the same question, the searched trajectory ranks higher before—and therefore after—group normalization exactly when

λg​di,j​\(1−bi\)−η​ci,j\>αcb​bi\.\\lambda\_\{g\}d\_\{i,j\}\(1\-b\_\{i\}\)\-\\eta c\_\{i,j\}\>\\alpha\_\{\\mathrm\{cb\}\}b\_\{i\}\.\(7\)Thus increasing closed\-book sufficiency shifts credit from searched to direct behavior, while evidence reliance and retrieval necessity must jointly offset search cost\. More generally, ifrinv<rwrong<1−ηr\_\{\\mathrm\{inv\}\}<r\_\{\\mathrm\{wrong\}\}<1\-\\eta,η∈\[0,1\)\\eta\\in\[0,1\), andλg,αcb∈\[0,1\]\\lambda\_\{g\},\\alpha\_\{\\mathrm\{cb\}\}\\in\[0,1\], the shaping preserves correctness\-first ordering: every correct valid trajectory outranks every wrong valid trajectory, which in turn outranks every invalid output\.

### 3\.4Masked GRPO Update

GRPO converts each trajectory reward into a group\-relative advantage,

A^i,j=ri,j−μiσi\+δ,\\hat\{A\}\_\{i,j\}=\\frac\{r\_\{i,j\}\-\\mu\_\{i\}\}\{\\sigma\_\{i\}\+\\delta\},\(8\)whereμi\\mu\_\{i\}andσi\\sigma\_\{i\}are the group mean and sample standard deviation andδ\>0\\delta\>0stabilizes normalization\.

We optimize only model\-generated reasoning, search actions, and final\-answer tokens\. LetMi,j,t=1M\_\{i,j,t\}=1for these tokens and0for prompt tokens, retrieved observations, and padding\. For a selected tokenwi,j,tw\_\{i,j,t\}, define

ρi,j,t=πθ​\(wi,j,t∣τi,j,<t\)πold​\(wi,j,t∣τi,j,<t\)\.\\rho\_\{i,j,t\}=\\frac\{\\pi\_\{\\theta\}\(w\_\{i,j,t\}\\mid\\tau\_\{i,j,<t\}\)\}\{\\pi\_\{\\mathrm\{old\}\}\(w\_\{i,j,t\}\\mid\\tau\_\{i,j,<t\}\)\}\.\(9\)Writing⟨⋅⟩ℳ,ℬ\\langle\\cdot\\rangle\_\{\\mathcal\{M\},\\mathcal\{B\}\}for the distributed masked average defined in[Equation˜12](https://arxiv.org/html/2608.07531#S7.E12), Search\-G1 maximizes

ℓi,j,tclip=\\displaystyle\\ell^\{\\mathrm\{clip\}\}\_\{i,j,t\}=\{\}min⁡\{ρi,j,t​A^i,j,clip⁡\(ρi,j,t,1−ϵ,1\+ϵ\)​A^i,j\},\\displaystyle\\min\\\!\\left\\\{\\rho\_\{i,j,t\}\\hat\{A\}\_\{i,j\},\\operatorname\{clip\}\(\\rho\_\{i,j,t\},1\-\\epsilon,1\+\\epsilon\)\\hat\{A\}\_\{i,j\}\\right\\\},\(10\)𝒥G1​\(θ\)=\\displaystyle\\mathcal\{J\}\_\{\\mathrm\{G1\}\}\(\\theta\)=\{\}⟨ℓi,j,tclip\+αHℋ\(πθ\(⋅∣τi,j,<t\)\)−βKLKL^i,j,tLV⟩ℳ,ℬ\.\\displaystyle\\left\\langle\\ell^\{\\mathrm\{clip\}\}\_\{i,j,t\}\+\\alpha\_\{\\mathrm\{H\}\}\\mathcal\{H\}\(\\pi\_\{\\theta\}\(\\cdot\\mid\\tau\_\{i,j,<t\}\)\)\-\\beta\_\{\\mathrm\{KL\}\}\\widehat\{\\mathrm\{KL\}\}^\{\\mathrm\{LV\}\}\_\{i,j,t\}\\right\\rangle\_\{\\mathcal\{M\},\\mathcal\{B\}\}\.Hereϵ\\epsilonis the clipping width;αH\\alpha\_\{\\mathrm\{H\}\}andβKL\\beta\_\{\\mathrm\{KL\}\}weight entropy and the low\-variance sampled\-token KL penalty, respectively; andπref\\pi\_\{\\mathrm\{ref\}\}is the reference policy used by that penalty\. Its exact estimator is defined in[Equation˜13](https://arxiv.org/html/2608.07531#S7.E13)\.

##### Effect of GRPO Normalization\.

To isolate how normalization affects the question\-level gate, let an analysis coefficientγ∈\[0,1\]\\gamma\\in\[0,1\]replacenin\_\{i\}only in the correct\-searched reliance term, withbib\_\{i\}and every other reward branch held fixed\. Thusγ=ni\\gamma=n\_\{i\}recovers Search\-G1 andγ=1\\gamma=1gives the controlled no\-gate comparison\. Letxi,j=di,jx\_\{i,j\}=d\_\{i,j\}for valid correct searched trajectories and0otherwise, with𝐱i=\(xi,1,…,xi,G\)\\mathbf\{x\}\_\{i\}=\(x\_\{i,1\},\\ldots,x\_\{i,G\}\)\. Setting𝐚i=𝐫i​\(0\)\\mathbf\{a\}\_\{i\}=\\mathbf\{r\}\_\{i\}\(0\)gives𝐫i​\(γ\)=𝐚i\+λg​γ​𝐱i\\mathbf\{r\}\_\{i\}\(\\gamma\)=\\mathbf\{a\}\_\{i\}\+\\lambda\_\{g\}\\gamma\\mathbf\{x\}\_\{i\}\. Let𝐮i\\mathbf\{u\}\_\{i\}and𝐯i\\mathbf\{v\}\_\{i\}be the group\-centered versions of𝐚i\\mathbf\{a\}\_\{i\}and𝐱i\\mathbf\{x\}\_\{i\}, and define𝐳i​\(γ\)=𝐮i\+λg​γ​𝐯i\\mathbf\{z\}\_\{i\}\(\\gamma\)=\\mathbf\{u\}\_\{i\}\+\\lambda\_\{g\}\\gamma\\mathbf\{v\}\_\{i\}\.

###### Proposition 1\(Effect of the necessity gate under GRPO normalization\)\.

Assumeλg\>0\\lambda\_\{g\}\>0\. For anyγ∈\[0,1\]\\gamma\\in\[0,1\]with𝐳i​\(γ\)≠𝟎\\mathbf\{z\}\_\{i\}\(\\gamma\)\\neq\\mathbf\{0\}and anyδ≥0\\delta\\geq 0, the normalized advantage vector is

𝐀iδ​\(γ\)=𝐳i​\(γ\)‖𝐳i​\(γ\)‖2/G−1\+δ\.\\mathbf\{A\}\_\{i\}^\{\\delta\}\(\\gamma\)=\\frac\{\\mathbf\{z\}\_\{i\}\(\\gamma\)\}\{\\\|\\mathbf\{z\}\_\{i\}\(\\gamma\)\\\|\_\{2\}/\\sqrt\{G\-1\}\+\\delta\}\.\(11\)Normalization therefore preserves the direction of𝐳i​\(γ\)\\mathbf\{z\}\_\{i\}\(\\gamma\), and fixes its norm toG−1\\sqrt\{G\-1\}whenδ=0\\delta=0\. Forγ1≠γ2\\gamma\_\{1\}\\neq\\gamma\_\{2\}with𝐳i​\(γk\)≠𝟎\\mathbf\{z\}\_\{i\}\(\\gamma\_\{k\}\)\\neq\\mathbf\{0\}fork∈\{1,2\}k\\in\\\{1,2\\\}, linear independence of𝐮i\\mathbf\{u\}\_\{i\}and𝐯i\\mathbf\{v\}\_\{i\}implies distinct credit directions; if they are collinear, only a degenerate global reversal through zero is possible\. In a non\-degenerate all\-valid, all\-correct, all\-searched, equal\-cost group,𝐮i=𝟎\\mathbf\{u\}\_\{i\}=\\mathbf\{0\}, so everyγ\>0\\gamma\>0preserves rollout ranking, advantage signs, and credit direction\. The gate cancels exactly whenδ=0\\delta=0and changes only the global scale whenδ\>0\\delta\>0\.

The proof, strict homogeneous form, and boundary cases are given in[Section˜7\.1](https://arxiv.org/html/2608.07531#S7.SS1.SSS0.Px4)\.

##### Current\-Policy Recalibration\.

Policy updates change both search behavior and the hidden\-state geometry used by the readouts\. At each calibration round, Search\-G1 therefore freezes the latest checkpoint asθ¯m\\bar\{\\theta\}\_\{m\}, regenerates both intervention targets, and refits the two heads\. The resulting snapshot–head evaluator is held fixed over the following update interval, and current trajectories are teacher\-forced through it without gradient tracking before reward scoring\. Hereπold\\pi\_\{\\mathrm\{old\}\}is the batch\-local behavior policy for one rollout group, whereasθ¯m\\bar\{\\theta\}\_\{m\}persists across multiple updates as the reward evaluator\. The updated policy supplies the measurements for the next refresh, closing the measurement–credit–adaptation loop\. Calibration schedules and refresh\-point stability diagnostics are reported in[Sections˜7\.2\.1](https://arxiv.org/html/2608.07531#S7.SS2.SSS1)and[A21](https://arxiv.org/html/2608.07531#S7.T21)\.

## 4Evaluation

We evaluate Search\-G1 along three axes: task utility, grounding, and search\-marker use; readout fidelity under policy drift; and the effect of necessity\-conditioned reliance on group\-relative credit\. The main comparison reports task and behavioral outcomes, while the subsequent analyses separately test the readouts, normalized credit, adaptation, and reward components\. We assess the retained empirical pattern through concordant mean trends in EM, TC/D\-Inv, and Search/Q rather than collapsing them into a single score\.

### 4\.1Experimental Setup

##### Models\.

The primary evaluation uses Qwen2\.5\-3B\-Instruct and Qwen2\.5\-7B\-Instruct\[[37](https://arxiv.org/html/2608.07531#bib.bib31)\]; analyses restricted to one scale or dataset are explicitly scoped in their captions\.

##### Benchmarks and metrics\.

We organize the evaluation along the three column groups of[Table˜1](https://arxiv.org/html/2608.07531#S4.T1):*task utility*,*grounding*, and*search cost*\.*Task utility*is measured by exact match \(EM\) on NQ\[[22](https://arxiv.org/html/2608.07531#bib.bib32)\], HotpotQA\[[45](https://arxiv.org/html/2608.07531#bib.bib33)\], 2WikiMultiHopQA\[[17](https://arxiv.org/html/2608.07531#bib.bib37)\], and MuSiQue\[[38](https://arxiv.org/html/2608.07531#bib.bib38)\], covering single\-hop factual retrieval and multi\-hop evidence aggregation\.*Grounding*is quantified by two counterfactual metrics: trust consistency \(TC↑\\uparrow\), which summarizes counterfactual responses over correct direct and searched trajectories, and document invariance \(D\-Inv↓\\downarrow\), which isolates originally correct searched answers that remain unchanged after document replacement\. The appendix reports an intervention\-free human protocol that serves as a complementary check\.*Search cost*is reported by Search/Q↓\\downarrow, the mean number of response\-side parser\-observed search markers per question\. Because a marker is not itself a compute unit, we separately account policy\-generated and retrieved\-observation tokens in[Table˜A5](https://arxiv.org/html/2608.07531#S7.T5), and report the full held\-out closed\-book composition in[Table˜A3](https://arxiv.org/html/2608.07531#S7.T3)\.

Table 1:Overall comparison of search\-agent RL methods\. Entries report mean±\\pmSD over four repetitions, with standard deviations as gray subscripts\. Arrows show absolute changes fromPrompted Search; green/red follow the displayed metric directions\. TC is graded counterfactual\-document trust consistency, while D\-Inv is counterfactual\-document invariance among classified originally correct searched trajectories\. Both are operational grounding metrics and, along with Search/Q, are evaluated on NQ\.MethodSignalTask UtilityGroundingSearch CostNQ \(↑\\uparrow\)HotpotQA \(↑\\uparrow\)2WikiMultiHopQA \(↑\\uparrow\)MuSiQue \(↑\\uparrow\)TC \(↑\\uparrow\)D\-Inv \(↓\\downarrow\)Search/Q \(↓\\downarrow\)Qwen2\.5\-3B\-InstructBaseNone0\.1420\.1530\.2120\.029n/an/a0\.000Prompted SearchRetrieval0\.2500\.1990\.2380\.0570\.9300\.0692\.215Search\-R1\[[19](https://arxiv.org/html/2608.07531#bib.bib3)\]Outcome\+Direct0\.352±0\.0190\.352\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.019\}\}↑0\.102\\uparrow\\\!0\.1020\.248±0\.0150\.248\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.015\}\}↑0\.049\\uparrow\\\!0\.0490\.267±0\.0140\.267\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.029\\uparrow\\\!0\.0290\.087±0\.0120\.087\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.012\}\}↑0\.030\\uparrow\\\!0\.0300\.917±0\.0180\.917\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.018\}\}↓0\.013\\downarrow\\\!0\.0130\.049±0\.0110\.049\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.011\}\}↓0\.020\\downarrow\\\!0\.0203\.281±0\.213\.281\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.21\}\}↑1\.066\\uparrow\\\!1\.066GiGPO\[[7](https://arxiv.org/html/2608.07531#bib.bib42)\]State0\.422±0\.0210\.422\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.021\}\}↑0\.172\\uparrow\\\!0\.1720\.296±0\.0170\.296\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.017\}\}↑0\.097\\uparrow\\\!0\.0970\.330±0\.0160\.330\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.016\}\}↑0\.092\\uparrow\\\!0\.0920\.126±0\.0140\.126\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.069\\uparrow\\\!0\.0690\.935±0\.0150\.935\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.015\}\}↑0\.005\\uparrow\\\!0\.0050\.047±0\.0100\.047\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.010\}\}↓0\.022\\downarrow\\\!0\.0223\.131±0\.183\.131\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.18\}\}↑0\.916\\uparrow\\\!0\.916AgentPRM\[[44](https://arxiv.org/html/2608.07531#bib.bib5)\]Process0\.380±0\.0200\.380\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.020\}\}↑0\.130\\uparrow\\\!0\.1300\.293±0\.0180\.293\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.018\}\}↑0\.094\\uparrow\\\!0\.0940\.328±0\.0170\.328\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.017\}\}↑0\.090\\uparrow\\\!0\.0900\.112±0\.0130\.112\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.013\}\}↑0\.055\\uparrow\\\!0\.0550\.958±0\.0120\.958\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.012\}\}↑0\.028\\uparrow\\\!0\.0280\.042±0\.0090\.042\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.009\}\}↓0\.027\\downarrow\\\!0\.0273\.340±0\.233\.340\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.23\}\}↑1\.125\\uparrow\\\!1\.125GPT\-5 Judge\[[25](https://arxiv.org/html/2608.07531#bib.bib6),[33](https://arxiv.org/html/2608.07531#bib.bib45)\]External Judge0\.402±0\.0220\.402\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.022\}\}↑0\.152\\uparrow\\\!0\.1520\.280±0\.0180\.280\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.018\}\}↑0\.081\\uparrow\\\!0\.0810\.324±0\.0170\.324\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.017\}\}↑0\.086\\uparrow\\\!0\.0860\.101±0\.0140\.101\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.044\\uparrow\\\!0\.0440\.946±0\.0160\.946\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.016\}\}↑0\.016\\uparrow\\\!0\.0160\.038±0\.0080\.038\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.008\}\}↓0\.031\\downarrow\\\!0\.0313\.238±0\.223\.238\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.22\}\}↑1\.023\\uparrow\\\!1\.023ARPO\[[5](https://arxiv.org/html/2608.07531#bib.bib13)\]Entropy0\.389±0\.0180\.389\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.018\}\}↑0\.139\\uparrow\\\!0\.1390\.273±0\.0160\.273\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.016\}\}↑0\.074\\uparrow\\\!0\.0740\.304±0\.0150\.304\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.015\}\}↑0\.066\\uparrow\\\!0\.0660\.109±0\.0130\.109\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.013\}\}↑0\.052\\uparrow\\\!0\.0520\.924±0\.0170\.924\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.017\}\}↓0\.006\\downarrow\\\!0\.0060\.055±0\.0120\.055\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.012\}\}↓0\.014\\downarrow\\\!0\.0142\.679±0\.162\.679\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.16\}\}↑0\.464\\uparrow\\\!0\.464IGPO\[[41](https://arxiv.org/html/2608.07531#bib.bib14)\]Information Gain0\.414±0\.0210\.414\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.021\}\}↑0\.164\\uparrow\\\!0\.1640\.292±0\.0170\.292\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.017\}\}↑0\.093\\uparrow\\\!0\.0930\.355±0\.0160\.355\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.016\}\}↑0\.117\\uparrow\\\!0\.1170\.137±0\.0150\.137\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.015\}\}↑0\.080\\uparrow\\\!0\.0800\.934±0\.0160\.934\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.016\}\}↑0\.004\\uparrow\\\!0\.0040\.056±0\.0120\.056\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.012\}\}↓0\.013\\downarrow\\\!0\.0133\.028±0\.193\.028\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.19\}\}↑0\.813\\uparrow\\\!0\.813Search\-E1\[[23](https://arxiv.org/html/2608.07531#bib.bib48)\]Self\-distillation0\.437±0\.0200\.437\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.020\}\}↑0\.187\\uparrow\\\!0\.1870\.314±0\.017\\mathbf\{0\.314\}\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.017\}\}↑0\.115\\uparrow\\\!0\.1150\.372±0\.016\\mathbf\{0\.372\}\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.016\}\}↑0\.134\\uparrow\\\!0\.1340\.142±0\.0140\.142\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.085\\uparrow\\\!0\.0850\.939±0\.0150\.939\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.015\}\}↑0\.009\\uparrow\\\!0\.0090\.054±0\.0110\.054\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.011\}\}↓0\.015\\downarrow\\\!0\.0152\.980±0\.182\.980\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.18\}\}↑0\.765\\uparrow\\\!0\.765SAAS\[[36](https://arxiv.org/html/2608.07531#bib.bib8)\]Rollout Boundary0\.428±0\.0190\.428\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.019\}\}↑0\.178\\uparrow\\\!0\.1780\.301±0\.0170\.301\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.017\}\}↑0\.102\\uparrow\\\!0\.1020\.355±0\.0160\.355\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.016\}\}↑0\.117\\uparrow\\\!0\.1170\.138±0\.0140\.138\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.081\\uparrow\\\!0\.0810\.951±0\.0130\.951\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.013\}\}↑0\.021\\uparrow\\\!0\.0210\.045±0\.0100\.045\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.010\}\}↓0\.024\\downarrow\\\!0\.0242\.122±0\.162\.122\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.16\}\}↓0\.093\\downarrow\\\!0\.093β\\beta\-GRPO\[[43](https://arxiv.org/html/2608.07531#bib.bib24)\]Query Confidence0\.419±0\.0200\.419\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.020\}\}↑0\.169\\uparrow\\\!0\.1690\.295±0\.0170\.295\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.017\}\}↑0\.096\\uparrow\\\!0\.0960\.344±0\.0160\.344\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.016\}\}↑0\.106\\uparrow\\\!0\.1060\.129±0\.0140\.129\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.072\\uparrow\\\!0\.0720\.948±0\.0140\.948\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.018\\uparrow\\\!0\.0180\.047±0\.0100\.047\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.010\}\}↓0\.022\\downarrow\\\!0\.0222\.227±0\.152\.227\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.15\}\}↑0\.012\\uparrow\\\!0\.012KbPO\[[8](https://arxiv.org/html/2608.07531#bib.bib10)\]Certainty\+Quality0\.431±0\.0190\.431\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.019\}\}↑0\.181\\uparrow\\\!0\.1810\.305±0\.0160\.305\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.016\}\}↑0\.106\\uparrow\\\!0\.1060\.358±0\.0150\.358\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.015\}\}↑0\.120\\uparrow\\\!0\.1200\.140±0\.0130\.140\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.013\}\}↑0\.083\\uparrow\\\!0\.0830\.955±0\.0120\.955\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.012\}\}↑0\.025\\uparrow\\\!0\.0250\.043±0\.0090\.043\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.009\}\}↓0\.026\\downarrow\\\!0\.0262\.160±0\.172\.160\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.17\}\}↓0\.055\\downarrow\\\!0\.055Search\-G1 \(ours\)Representation0\.453±0\.018\\mathbf\{0\.453\}\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.018\}\}↑0\.203\\uparrow\\\!0\.2030\.306±0\.0160\.306\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.016\}\}↑0\.107\\uparrow\\\!0\.1070\.368±0\.0150\.368\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.015\}\}↑0\.130\\uparrow\\\!0\.1300\.143±0\.013\\mathbf\{0\.143\}\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.013\}\}↑0\.086\\uparrow\\\!0\.0860\.976±0\.010\\mathbf\{0\.976\}\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.010\}\}↑0\.046\\uparrow\\\!0\.0460\.034±0\.007\\mathbf\{0\.034\}\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.007\}\}↓0\.035\\downarrow\\\!0\.0352\.094±0\.14\\mathbf\{2\.094\}\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.14\}\}↓0\.121\\downarrow\\\!0\.121Qwen2\.5\-7B\-InstructBaseNone0\.1920\.1830\.2500\.031n/an/a0\.000Prompted SearchRetrieval0\.3490\.2990\.2350\.0580\.9360\.0612\.171Search\-R1\[[19](https://arxiv.org/html/2608.07531#bib.bib3)\]Outcome\+Direct0\.393±0\.0170\.393\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.017\}\}↑0\.044\\uparrow\\\!0\.0440\.370±0\.0140\.370\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.071\\uparrow\\\!0\.0710\.414±0\.0130\.414\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.013\}\}↑0\.179\\uparrow\\\!0\.1790\.146±0\.0110\.146\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.011\}\}↑0\.088\\uparrow\\\!0\.0880\.932±0\.0160\.932\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.016\}\}↓0\.004\\downarrow\\\!0\.0040\.046±0\.0100\.046\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.010\}\}↓0\.015\\downarrow\\\!0\.0153\.090±0\.193\.090\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.19\}\}↑0\.919\\uparrow\\\!0\.919GiGPO\[[7](https://arxiv.org/html/2608.07531#bib.bib42)\]State0\.464±0\.0190\.464\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.019\}\}↑0\.115\\uparrow\\\!0\.1150\.416±0\.0150\.416\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.015\}\}↑0\.117\\uparrow\\\!0\.1170\.436±0\.0140\.436\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.201\\uparrow\\\!0\.2010\.189±0\.0130\.189\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.013\}\}↑0\.131\\uparrow\\\!0\.1310\.947±0\.0140\.947\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.011\\uparrow\\\!0\.0110\.043±0\.0090\.043\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.009\}\}↓0\.018\\downarrow\\\!0\.0182\.930±0\.162\.930\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.16\}\}↑0\.759\\uparrow\\\!0\.759AgentPRM\[[44](https://arxiv.org/html/2608.07531#bib.bib5)\]Process0\.455±0\.0180\.455\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.018\}\}↑0\.106\\uparrow\\\!0\.1060\.405±0\.0160\.405\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.016\}\}↑0\.106\\uparrow\\\!0\.1060\.430±0\.0150\.430\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.015\}\}↑0\.195\\uparrow\\\!0\.1950\.205±0\.0120\.205\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.012\}\}↑0\.147\\uparrow\\\!0\.1470\.963±0\.0110\.963\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.011\}\}↑0\.027\\uparrow\\\!0\.0270\.037±0\.0080\.037\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.008\}\}↓0\.024\\downarrow\\\!0\.0243\.080±0\.213\.080\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.21\}\}↑0\.909\\uparrow\\\!0\.909GPT\-5 Judge\[[25](https://arxiv.org/html/2608.07531#bib.bib6),[33](https://arxiv.org/html/2608.07531#bib.bib45)\]External Judge0\.470±0\.0200\.470\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.020\}\}↑0\.121\\uparrow\\\!0\.1210\.410±0\.0160\.410\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.016\}\}↑0\.111\\uparrow\\\!0\.1110\.440±0\.0150\.440\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.015\}\}↑0\.205\\uparrow\\\!0\.2050\.215±0\.0130\.215\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.013\}\}↑0\.157\\uparrow\\\!0\.1570\.952±0\.0140\.952\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.016\\uparrow\\\!0\.0160\.033±0\.0070\.033\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.007\}\}↓0\.028\\downarrow\\\!0\.0282\.980±0\.202\.980\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.20\}\}↑0\.809\\uparrow\\\!0\.809ARPO\[[5](https://arxiv.org/html/2608.07531#bib.bib13)\]Entropy0\.462±0\.0160\.462\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.016\}\}↑0\.113\\uparrow\\\!0\.1130\.398±0\.0140\.398\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.099\\uparrow\\\!0\.0990\.420±0\.0140\.420\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.185\\uparrow\\\!0\.1850\.208±0\.0120\.208\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.012\}\}↑0\.150\\uparrow\\\!0\.1500\.931±0\.0150\.931\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.015\}\}↓0\.005\\downarrow\\\!0\.0050\.050±0\.0110\.050\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.011\}\}↓0\.011\\downarrow\\\!0\.0112\.430±0\.142\.430\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.14\}\}↑0\.259\\uparrow\\\!0\.259IGPO\[[41](https://arxiv.org/html/2608.07531#bib.bib14)\]Information Gain0\.480±0\.0190\.480\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.019\}\}↑0\.131\\uparrow\\\!0\.1310\.425±0\.0150\.425\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.015\}\}↑0\.126\\uparrow\\\!0\.1260\.455±0\.0140\.455\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.220\\uparrow\\\!0\.2200\.235±0\.0140\.235\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.177\\uparrow\\\!0\.1770\.942±0\.0140\.942\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.006\\uparrow\\\!0\.0060\.050±0\.0110\.050\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.011\}\}↓0\.011\\downarrow\\\!0\.0112\.760±0\.172\.760\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.17\}\}↑0\.589\\uparrow\\\!0\.589Search\-E1\[[23](https://arxiv.org/html/2608.07531#bib.bib48)\]Self\-distillation0\.495±0\.0180\.495\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.018\}\}↑0\.146\\uparrow\\\!0\.1460\.440±0\.0150\.440\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.015\}\}↑0\.141\\uparrow\\\!0\.1410\.465±0\.0140\.465\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.230\\uparrow\\\!0\.2300\.245±0\.0130\.245\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.013\}\}↑0\.187\\uparrow\\\!0\.1870\.947±0\.0140\.947\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.011\\uparrow\\\!0\.0110\.048±0\.0100\.048\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.010\}\}↓0\.013\\downarrow\\\!0\.0132\.720±0\.162\.720\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.16\}\}↑0\.549\\uparrow\\\!0\.549SAAS\[[36](https://arxiv.org/html/2608.07531#bib.bib8)\]Rollout Boundary0\.486±0\.0170\.486\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.017\}\}↑0\.137\\uparrow\\\!0\.1370\.432±0\.0150\.432\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.015\}\}↑0\.133\\uparrow\\\!0\.1330\.458±0\.0140\.458\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.223\\uparrow\\\!0\.2230\.239±0\.0130\.239\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.013\}\}↑0\.181\\uparrow\\\!0\.1810\.958±0\.0120\.958\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.012\}\}↑0\.022\\uparrow\\\!0\.0220\.038±0\.0080\.038\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.008\}\}↓0\.023\\downarrow\\\!0\.0232\.050±0\.152\.050\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.15\}\}↓0\.121\\downarrow\\\!0\.121β\\beta\-GRPO\[[43](https://arxiv.org/html/2608.07531#bib.bib24)\]Query Confidence0\.478±0\.0180\.478\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.018\}\}↑0\.129\\uparrow\\\!0\.1290\.424±0\.0150\.424\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.015\}\}↑0\.125\\uparrow\\\!0\.1250\.448±0\.0140\.448\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.213\\uparrow\\\!0\.2130\.231±0\.0130\.231\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.013\}\}↑0\.173\\uparrow\\\!0\.1730\.953±0\.0130\.953\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.013\}\}↑0\.017\\uparrow\\\!0\.0170\.040±0\.0090\.040\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.009\}\}↓0\.021\\downarrow\\\!0\.0212\.164±0\.142\.164\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.14\}\}↓0\.007\\downarrow\\\!0\.007KbPO\[[8](https://arxiv.org/html/2608.07531#bib.bib10)\]Certainty\+Quality0\.490±0\.0170\.490\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.017\}\}↑0\.141\\uparrow\\\!0\.1410\.436±0\.0140\.436\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.137\\uparrow\\\!0\.1370\.462±0\.0140\.462\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.227\\uparrow\\\!0\.2270\.242±0\.0120\.242\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.012\}\}↑0\.184\\uparrow\\\!0\.1840\.962±0\.0110\.962\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.011\}\}↑0\.026\\uparrow\\\!0\.0260\.036±0\.0080\.036\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.008\}\}↓0\.025\\downarrow\\\!0\.0252\.106±0\.152\.106\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.15\}\}↓0\.065\\downarrow\\\!0\.065Search\-G1 \(ours\)Representation0\.512±0\.016\\mathbf\{0\.512\}\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.016\}\}↑0\.163\\uparrow\\\!0\.1630\.450±0\.014\\mathbf\{0\.450\}\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.151\\uparrow\\\!0\.1510\.475±0\.014\\mathbf\{0\.475\}\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.014\}\}↑0\.240\\uparrow\\\!0\.2400\.254±0\.012\\mathbf\{0\.254\}\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.012\}\}↑0\.196\\uparrow\\\!0\.1960\.981±0\.009\\mathbf\{0\.981\}\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.009\}\}↑0\.045\\uparrow\\\!0\.0450\.028±0\.006\\mathbf\{0\.028\}\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.006\}\}↓0\.033\\downarrow\\\!0\.0331\.860±0\.13\\mathbf\{1\.860\}\_\{\\scriptscriptstyle\\color\[rgb\]\{0\.3984375,0\.3984375,0\.3984375\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.3984375,0\.3984375,0\.3984375\}\{\\pm 0\.13\}\}↓0\.311\\downarrow\\\!0\.311

##### Baselines\.

We compare with two non\-trained references:Base, the closed\-book model without retrieval, andPrompted Search, which uses the same retrieval interface without policy updates\. The training baselines cover complementary credit mechanisms:Search\-R1\[[19](https://arxiv.org/html/2608.07531#bib.bib3)\]optimizes terminal correctness with a direct\-answer bonus;GiGPO\[[7](https://arxiv.org/html/2608.07531#bib.bib42)\]adds anchor\-state groups for stepwise relative credit;AgentPRM\[[44](https://arxiv.org/html/2608.07531#bib.bib5)\]uses an external process reward model for step\-level progress;GPT\-5 Judge\[[33](https://arxiv.org/html/2608.07531#bib.bib45),[25](https://arxiv.org/html/2608.07531#bib.bib6)\]scores completed trajectories with an external evidence\-support judge;ARPO\[[5](https://arxiv.org/html/2608.07531#bib.bib13)\]uses post\-tool entropy for adaptive rollouts and attribution;IGPO\[[41](https://arxiv.org/html/2608.07531#bib.bib14)\]rewards turn\-level gains in ground\-truth confidence; andSearch\-E1\[[23](https://arxiv.org/html/2608.07531#bib.bib48)\]alternates GRPO with offline self\-distillation from sibling contexts\. We additionally compare with three closest boundary\-oriented methods\.SAAS\[[36](https://arxiv.org/html/2608.07531#bib.bib8)\]contrasts search\-disabled and search\-enabled rollout groups to infer an evolving boundary and penalize unnecessary or redundant search\.β\\beta\-GRPO\[[43](https://arxiv.org/html/2608.07531#bib.bib24)\]gates correctness reward by the minimum policy confidence over generated query tokens\.KbPO\[[8](https://arxiv.org/html/2608.07531#bib.bib10)\]combines parametric certainty with retrieval quality in a four\-quadrant reward\. Because their official pipelines differ from ours, we reproduce these methods in adapted form under the shared model, retrieval environment, and update budget; the retained components and deviations are reported in[Table˜A1](https://arxiv.org/html/2608.07531#S7.T1)\.

##### Implementation details\.

All trained methods share the policy\-training data, base model, retrieval interface, trajectory format, and global interaction/update cap\. Method\-specific process\-model inference, judge calls, likelihood passes, self\-distillation, and readout refitting are retained, so the comparison matches on\-policy exposure rather than end\-to\-end compute \([Table˜A1](https://arxiv.org/html/2608.07531#S7.T1)\)\. Results are means over four independently seeded training runs; Base and Prompted Search are untrained greedy references\. Optimization, parser, readout\-refresh, auxiliary\-compute, and evaluation settings are reported in[Section˜7\.2\.1](https://arxiv.org/html/2608.07531#S7.SS2.SSS1)\.

### 4\.2Main results

As shown in[Table˜1](https://arxiv.org/html/2608.07531#S4.T1), Search\-G1 attains the strongest overall profile across four benchmarks and two scales\. On task utility, it improves EM over Prompted Search by 0\.086–0\.203 \(3B\) and 0\.151–0\.240 \(7B\), and reaches the best or statistically comparable accuracy relative to the strongest RL baseline \(Search\-E1\)\. At the same time it attains the highest TC at both scales \(\+0\.018\+0\.018over the strongest alternative\) and the lowest D\-Inv \(−0\.004/−0\.005\-0\.004/\-0\.005\), while issuing 0\.121/0\.311 fewer search markers than Prompted Search; as the hierarchical test below shows, the D\-Inv and search\-use advantages over the strongest per\-metric baseline are significant at 7B but not at 3B\. The task\-accuracy parity and the grounding and efficiency gains are examined jointly below\.

##### Hierarchical inference\.

To confirm these differences are not seed artifacts,[Table˜2](https://arxiv.org/html/2608.07531#S4.T2)propagates both sources of variation on the four seed\-by\-question matrices—resampling training seeds first and paired questions within each seed—and applies Holm correction across the four endpoints within each comparator–scale family\. Against Search\-R1, all four endpoints are significant at both scales\. Against the strongest per\-metric baseline, the TC advantage survives at both scales; the D\-Inv and Search/Q advantages are significant at 7B but not at 3B, and EM is not statistically separable at either scale\.

Table 2:NQ hierarchical seed×\\timesquestion inference\. Each cell is difference \[hierarchical\-bootstrap 95% CI\]; Holm\-adjustedpp\. D\-Inv is comparator minus Search\-G1; negativeΔ\\DeltaSearch/Q favors Search\-G1\.†\\daggermarks adjustedp<0\.05p<0\.05\.ComparatorScaleΔ\\DeltaEMΔ\\DeltaTCD\-Inv benefitΔ\\DeltaSearch/QSearch\-R13B\+0\.101​\[\+0\.071,\+0\.131\];\.004†\+0\.101\\,\[\+0\.071,\+0\.131\];\\ \.004^\{\\dagger\}\+0\.059​\[\+0\.038,\+0\.080\];\.004†\+0\.059\\,\[\+0\.038,\+0\.080\];\\ \.004^\{\\dagger\}\+0\.015​\[\+0\.005,\+0\.025\];\.012†\+0\.015\\,\[\+0\.005,\+0\.025\];\\ \.012^\{\\dagger\}−1\.187​\[−1\.440,−0\.934\];\.004†\-1\.187\\,\[\-1\.440,\-0\.934\];\\ \.004^\{\\dagger\}7B\+0\.119​\[\+0\.091,\+0\.147\];\.004†\+0\.119\\,\[\+0\.091,\+0\.147\];\\ \.004^\{\\dagger\}\+0\.049​\[\+0\.030,\+0\.068\];\.004†\+0\.049\\,\[\+0\.030,\+0\.068\];\\ \.004^\{\\dagger\}\+0\.018​\[\+0\.008,\+0\.028\];\.008†\+0\.018\\,\[\+0\.008,\+0\.028\];\\ \.008^\{\\dagger\}−1\.230​\[−1\.460,−1\.000\];\.004†\-1\.230\\,\[\-1\.460,\-1\.000\];\\ \.004^\{\\dagger\}Strongest per\-metric3B\+0\.016​\[−0\.012,\+0\.044\];\.320\+0\.016\\,\[\-0\.012,\+0\.044\];\\ \.320\+0\.018​\[\+0\.004,\+0\.032\];\.032†\+0\.018\\,\[\+0\.004,\+0\.032\];\\ \.032^\{\\dagger\}\+0\.004​\[−0\.003,\+0\.011\];\.320\+0\.004\\,\[\-0\.003,\+0\.011\];\\ \.320−0\.028​\[−0\.114,\+0\.058\];\.520\-0\.028\\,\[\-0\.114,\+0\.058\];\\ \.5207B\+0\.017​\[−0\.006,\+0\.040\];\.180\+0\.017\\,\[\-0\.006,\+0\.040\];\\ \.180\+0\.018​\[\+0\.006,\+0\.030\];\.016†\+0\.018\\,\[\+0\.006,\+0\.030\];\\ \.016^\{\\dagger\}\+0\.005​\[\+0\.001,\+0\.009\];\.048†\+0\.005\\,\[\+0\.001,\+0\.009\];\\ \.048^\{\\dagger\}−0\.190​\[−0\.286,−0\.094\];\.008†\-0\.190\\,\[\-0\.286,\-0\.094\];\\ \.008^\{\\dagger\}

##### Multi\-dataset grounding\.

Because the grounding metrics in[Table˜1](https://arxiv.org/html/2608.07531#S4.T1)are computed on NQ, we replicate the same counterfactual metrics on 2WikiMultiHopQA\. As shown in[Table˜3](https://arxiv.org/html/2608.07531#S4.T3), Search\-G1 again attains the highest TC, the lowest D\-Inv, and fewer search markers than the Search\-R1 baseline at both scales, confirming that the grounding gains are not dataset\-specific\.

Table 3:Counterfactual grounding on 2WikiMultiHopQA \(four\-seed means\), complementing the NQ metrics in[Table˜1](https://arxiv.org/html/2608.07531#S4.T1)\.MethodEM↑\\uparrowTC↑\\uparrowD\-Inv↓\\downarrowSearch/Q↓\\downarrowQwen2\.5\-3B\-InstructPrompted Search0\.2380\.8770\.0852\.951Search\-R10\.2670\.9130\.0583\.402Search\-G10\.3680\.9720\.0312\.812Qwen2\.5\-7B\-InstructPrompted Search0\.2350\.8870\.0792\.892Search\-R10\.4140\.9210\.0503\.198Search\-G10\.4750\.9780\.0262\.495

### 4\.3Analysis

The following analyses trace the reward mechanism end to end: whether the estimator separates evidence reliance \(fidelity\), the credit it induces once GRPO normalization is applied \(tie\-breaking\), and whether it stays reliable as the policy drifts\.

##### Representation estimator fidelity\.

The representation estimator reaches AUC 0\.959/0\.971 on NQ and 0\.871/0\.914 on 2Wiki for 3B/7B \(four\-seed SD≤0\.014\\leq 0\.014\), well above the strongest logit readout \(0\.638–0\.731;[Figure˜3](https://arxiv.org/html/2608.07531#S4.F3)\); every setting keeps a\>0\.15\>0\.15AUC gap across seeds\. Layer localization, label\-reliability audits, and calibration are reported in[Figures˜A1](https://arxiv.org/html/2608.07531#S7.F1),[A6](https://arxiv.org/html/2608.07531#S7.T6),[A16](https://arxiv.org/html/2608.07531#S7.T16)and[7\.3\.4](https://arxiv.org/html/2608.07531#S7.SS3.SSS4.Px6)\.

![Refer to caption](https://arxiv.org/html/2608.07531v1/x1.png)Figure 3:Evidence\-reliance estimator fidelity for separating deletion\-sensitive from deletion\-invariant correct trajectories on NQ and 2WikiMultiHopQA using Qwen2\.5\-3B and Qwen2\.5\-7B\.
##### Group\-relative variation in all\-correct groups\.

In all\-correct groups, Search\-G1 restores nonzero reward variation in 46\.6–48\.7% of NQ groups and 91\.8–92\.4% of 2Wiki groups, with within\-group reward SDs of 0\.189–0\.305 \([Figure˜4](https://arxiv.org/html/2608.07531#S4.F4)\)\. Permutation and cost\-only controls are reported in[Table˜A22](https://arxiv.org/html/2608.07531#S7.T22)\.

![Refer to caption](https://arxiv.org/html/2608.07531v1/x2.png)Figure 4:Reward tie\-breaking in all\-correct GRPO groups\. Each stem begins at the EM\-only baseline of zero and ends at the corresponding Search\-G1 value\. \(a\) Fraction of groups with nonzero within\-group reward variation\. \(b\) Mean raw within\-group reward standard deviation\. Group counts \(3B/7B\) are 251/264 for NQ and 276/282 for 2Wiki\.
##### Representation drift and estimator adaptation\.

At checkpoint 150, refitted evidence\-reliance and retrieval\-necessity readouts retain AUC 0\.786–0\.891, versus 0\.482–0\.608 when frozen \(gaps 0\.283–0\.304; four\-seed SD≤0\.021\\leq 0\.021frozen and≤0\.015\\leq 0\.015refit, well below the gap;[Figure˜5](https://arxiv.org/html/2608.07531#S4.F5)\); refresh\-point stability is reported in[Table˜A21](https://arxiv.org/html/2608.07531#S7.T21)\.

![Refer to caption](https://arxiv.org/html/2608.07531v1/x3.png)Figure 5:Estimator fidelity across RL checkpoints\. Frozen estimators are trained at checkpoint 0 and applied unchanged, whereas Refit estimators are retrained at each checkpoint using the same probe architecture and data protocol\. Markers report held\-out AUC, and the horizontal dashed line denotes chance performance \(0\.5\)\.
##### Policy\-boundary behavior\.

The boundary analysis reports both a common Base\-defined split and method\-relative sensitivity analyses in[Tables˜A7](https://arxiv.org/html/2608.07531#S7.T7)and[A8](https://arxiv.org/html/2608.07531#S7.T8); the full held\-out population counts are given in[Table˜A3](https://arxiv.org/html/2608.07531#S7.T3)\.

##### Real retrieval and token cost\.

Re\-running the released checkpoints on the same NQ held\-out split \(greedy decoding, matched interaction budget\) with token\-level logging yields measured per\-question costs \([Table˜A4](https://arxiv.org/html/2608.07531#S7.T4); 7B in[Table˜A5](https://arxiv.org/html/2608.07531#S7.T5)\)\. Relative to Search\-R1, Search\-G1 cuts policy\-generated tokens per question by≈43%\\approx 43\\%\(149→85149\\\!\\to\\\!85at 3B\) and total response\-side tokens by≈16%\\approx 16\\%, while at the same time attaining substantially higher EM \(0\.352→0\.4530\.352\\\!\\to\\\!0\.453\) and emitting fewer response\-side search markers \(3\.281→2\.0943\.281\\\!\\to\\\!2\.094\) under a matched interaction budget\. Search\-G1 thus delivers markedly shorter response\-side trajectories together with stronger accuracy and grounding per unit of interaction\.

##### Additional checks\.

Two further checks corroborate these findings\. Under cross\-dataset transfer against a source\-matched Search\-R1 baseline, Search\-G1 improves in\-domain accuracy and matches the baseline off\-distribution without retraining, and a same\-question human evaluation confirms significantly higher evidence support\. Full aggregates, confidence intervals, and paired row\-level records are reported in[Tables˜A23](https://arxiv.org/html/2608.07531#S7.T23)and[A11](https://arxiv.org/html/2608.07531#S7.T11)\.

### 4\.4Ablations

##### Component ablations\.

We ablate Search\-G1 on 2WikiMultiHopQA, where multi\-hop answering requires both evidence use and controlled retrieval\. As shown in[Table˜4](https://arxiv.org/html/2608.07531#S4.T4), we compare the full method with frozen readouts and variants that remove the searched\-branch necessity gate, repeated\-search cost, or evidence\-reliance shaping\. The resulting component ordering shows that removing the readouts, reliance shaping, necessity assignment, or repeated\-search cost worsens at least one pre\-specified outcome without improving the others enough to dominate the full method; in particular, removing reliance shaping lowers Search/Q only because the policy searches less overall, at the cost of a comparably large drop in TC, rather than reflecting a genuine efficiency gain\. Entries are four\-run means and are read as relative comparisons rather than causal\-effect estimates\.

Table 4:Component and necessity ablations on 2WikiMultiHopQA\. Each cell reports EM / TC / Search/Q as four\-run means\.VariantQwen2\.5\-3BQwen2\.5\-7BFull Search\-G10\.368 / 0\.972 / 2\.8120\.475 / 0\.978 / 2\.495Frozen readouts0\.357 / 0\.906 / 3\.2250\.466 / 0\.922 / 2\.846w/o search cost0\.338 / 0\.920 / 3\.5760\.449 / 0\.934 / 3\.144w/o reliance shaping0\.354 / 0\.797 / 1\.9920\.463 / 0\.829 / 1\.798w/o searched gate0\.347 / 0\.877 / 3\.1190\.444 / 0\.897 / 2\.756w/o direct bonus0\.352 / 0\.915 / 2\.9510\.451 / 0\.936 / 2\.612w/o both necessity terms0\.339 / 0\.861 / 3\.2860\.436 / 0\.881 / 2\.905Shuffled searched gate0\.341 / 0\.869 / 3\.2040\.439 / 0\.889 / 2\.838

##### Necessity factorization\.

The main no\-gate ablation replacesnin\_\{i\}by11only in the searched branch and retains the question\-matched direct sufficiency bonusbib\_\{i\}, so it does not remove the complete necessity signal\. The rerun therefore separately removes each branch and permutesnin\_\{i\}across questions while preserving its marginal distribution\. Becausebib\_\{i\}remains matched to its original question, the shuffle tests searched\-branch assignment rather than all uses of necessity; exact definitions are in[Section˜7\.1](https://arxiv.org/html/2608.07531#S7.SS1.SSS0.Px3)\.

##### Reward\-coefficient sensitivity\.

The coefficient sweep in[Table˜A13](https://arxiv.org/html/2608.07531#S7.T13)shows a smooth task–grounding–search\-use trade\-off rather than an isolated default optimum\.

##### Readout\-update sensitivity\.

The update\-schedule comparison in[Table˜A14](https://arxiv.org/html/2608.07531#S7.T14)contrasts frozen, nominal, less frequent, and data\-limited refits under the same seed set and evaluation questions\.

##### Cost\-controlled comparison\.

The comparison in[Table˜A15](https://arxiv.org/html/2608.07531#S7.T15)scores all methods on one fixed 1,024\-question set under a unified protocol \(identical question IDs, invalid trajectories charged EM=0\{=\}0, and Search/Q computed over all questions\); EM per marker is reported as a secondary descriptive quantity, and measured generated and observation tokens are in[Tables˜A4](https://arxiv.org/html/2608.07531#S7.T4)and[A5](https://arxiv.org/html/2608.07531#S7.T5)\.

### 4\.5Case Study

A qualitative 2WikiMultiHopQA case study summarizes representative differences in answer accuracy, evidence alignment, and search behavior \([Figure˜A2](https://arxiv.org/html/2608.07531#S7.F2)\)\. The cases are selected illustrations rather than population\-level comparative evidence\.

## 5Conclusion

In this paper, we presentedSearch\-G1, an intervention\-calibrated, representation\-level intrinsic reward for selective grounded search\. Rather than relying on external judges or confidence\-based internal signals, it treats training as adaptive measurement: two*policy\-relative*readouts probe the current policy for retrieval necessity \(the closed\-book sufficiency gap\) and answer\-stage evidence reliance \(sensitivity to deleting retrieved evidence\), and their multiplicative gateg=d​\(1−b\)g=d\(1\-b\)shapes a piecewise, correctness\-first trajectory reward that credits grounded behavior without overriding task correctness\. Across four QA benchmarks and two scales, Search\-G1 improves grounding and shortens trajectories while keeping accuracy competitive, requiring no external judge calls during optimization at the cost of periodic calibration compute\.

## 6Limitations

Our evaluation focuses on text\-retrieval QA with two Qwen backbones, a fixed retrieval environment, and parser\-observed search markers; extending the study to other model families, retrievers, richer tools, long\-horizon web interaction, and non\-textual evidence is left to future work\. Counterfactual grounding metrics and evidence\-deletion estimator audits are computed under a single fixed retriever and intervention design, so their absolute levels can shift with retriever quality even though the relative comparisons are held on identical inputs\. Search/Q captures model marker\-generation behavior, and our cost audit locates the efficiency gain in policy\-generated and total response\-side tokens; we therefore frame the benefit as a lower trajectory/token cost rather than a wall\-clock or monetary\-cost claim\. Periodic readout refitting adds calibration compute whose end\-to\-end overhead depends on hardware and batching\. Broader matched human evaluation, more diverse retrieval environments, and larger\-scale training are natural next steps for further testing the generality of our findings\.

## References

- \[1\]\(2023\)The internal state of an llm knows when it’s lying\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 967–976\.Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[2\]R\. Cheng, H\. Ma, Z\. Hai, Y\. Huang, R\. Duan, T\. Zhang, X\. Yang, Z\. Ye, and X\. Ma\(2026\)Adversarial orthogonal disentanglement for lvlm hallucination mitigation\.arXiv preprint arXiv:2605\.25377\.Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[3\]R\. Cheng, H\. Ma, W\. Wang, R\. Duan, J\. Liu, X\. Jia, S\. Qin, X\. Cao, Y\. Liu, and X\. Jia\(2025\)Inverse reinforcement learning with dynamic reward scaling for llm alignment\.arXiv preprint arXiv:2503\.18991\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p1.1)\.
- \[4\]Y\. Chuang, Y\. Xie, H\. Luo, Y\. Kim, J\. R\. Glass, and P\. He\(2024\)Dola: decoding by contrasting layers improves factuality in large language models\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 54158–54183\.Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[5\]G\. Dong, H\. Mao, K\. Ma, L\. Bao, Y\. Chen, Z\. Wang, Z\. Chen, J\. Du, H\. Wang, F\. Zhang,et al\.\(2025\)Agentic reinforced policy optimization\.arXiv preprint arXiv:2507\.19849\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.07531#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.234.232.232.15),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.79.77.77.15)\.
- \[6\]Y\. Du, L\. Han, M\. Fang, J\. Liu, T\. Dai, and D\. Tao\(2019\)Liir: learning individual intrinsic reward in multi\-agent reinforcement learning\.Advances in neural information processing systems32\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p3.1)\.
- \[7\]L\. Feng, Z\. Xue, T\. Liu, and B\. An\(2026\)Group\-in\-group policy optimization for llm agent training\.Advances in Neural Information Processing Systems38,pp\. 46375–46408\.Cited by:[§2\.1](https://arxiv.org/html/2608.07531#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.192.190.190.15.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.37.35.35.15.1)\.
- \[8\]T\. Feng, X\. Jiang, X\. Hu, Y\. Zhang, Z\. Tao, W\. Zhang, B\. Liu, W\. Jiang, and C\. Wu\(2026\-07\)Trust within? seek beyond? knowledge boundary aware policy optimization for agentic search\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 27664–27682\.External Links:[Link](https://aclanthology.org/2026.acl-long.1276/),[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.1276),ISBN 979\-8\-89176\-390\-6Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p1.1),[§2\.1](https://arxiv.org/html/2608.07531#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.150.148.148.15.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.305.303.303.15.1)\.
- \[9\]T\. Feng, X\. Jiang, and C\. Wu\(2026\)KbSD: knowledge boundary aware self\-distillation for behavioral calibration in agentic search\.arXiv preprint arXiv:2606\.29863\.Cited by:[§2\.1](https://arxiv.org/html/2608.07531#S2.SS1.p1.1)\.
- \[10\]J\. Ferrando, O\. Obeso, S\. Rajamanoharan, and N\. Nanda\(2025\)Do i know this entity? knowledge awareness and hallucinations in language models\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 77965–78000\.Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[11\]J\. Gao, W\. Fu, M\. Xie, S\. Xu, C\. He, Z\. Mei, B\. Zhu, and Y\. Wu\(2025\)Beyond ten turns: unlocking long\-horizon agentic search with large\-scale asynchronous rl\.arXiv preprint arXiv:2508\.07976\.Cited by:[§2\.1](https://arxiv.org/html/2608.07531#S2.SS1.p1.1)\.
- \[12\]M\. Geva, R\. Schuster, J\. Berant, and O\. Levy\(2021\)Transformer feed\-forward layers are key\-value memories\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,pp\. 5484–5495\.Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[13\]A\. Giloni, C\. Picardi, R\. Betser, S\. Bose, A\. P\. R\. Sabapathy, and R\. Vainshtein\(2025\)CAIR: counterfactual\-based agent influence ranker for agentic ai workflows\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 18951–18977\.Cited by:[§2\.1](https://arxiv.org/html/2608.07531#S2.SS1.p1.1)\.
- \[14\]D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi,et al\.\(2025\)Deepseek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p1.1)\.
- \[15\]K\. Healy, B\. Srinivasan, V\. Madathil, and J\. Wu\(2026\)Internal representations as indicators of hallucinations in agent tool selection\.arXiv preprint arXiv:2601\.05214\.Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[16\]E\. Hernandez, B\. Z\. Li, and J\. Andreas\(2023\)Inspecting and editing knowledge representations in language models\.arXiv preprint arXiv:2304\.00740\.Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[17\]X\. Ho, A\. Duong Nguyen, S\. Sugawara, and A\. Aizawa\(2020\-12\)Constructing a multi\-hop QA dataset for comprehensive evaluation of reasoning steps\.InProceedings of the 28th International Conference on Computational Linguistics,D\. Scott, N\. Bel, and C\. Zong \(Eds\.\),Barcelona, Spain \(Online\),pp\. 6609–6625\.External Links:[Link](https://aclanthology.org/2020.coling-main.580/),[Document](https://dx.doi.org/10.18653/v1/2020.coling-main.580)Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p2.4),[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px2.p1.3)\.
- \[18\]Z\. Hu, Z\. Shi, M\. Zhu, H\. Li, T\. Sun, P\. Ren, S\. Verberne, and Z\. Ren\(2025\)OpenReward: learning to reward long\-form agentic tasks via reinforcement learning\.arXiv preprint arXiv:2510\.24636\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p3.1)\.
- \[19\]B\. Jin, H\. Zeng, Z\. Yue, J\. Yoon, S\. Arik, D\. Wang, H\. Zamani, and J\. Han\(2025\)Search\-r1: training llms to reason and leverage search engines with reinforcement learning\.arXiv preprint arXiv:2503\.09516\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p1.1),[§1](https://arxiv.org/html/2608.07531#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.07531#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.178.176.176.15),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.23.21.21.15)\.
- \[20\]S\. Kadavath, T\. Conerly, A\. Askell, T\. Henighan, D\. Drain, E\. Perez, N\. Schiefer, Z\. Hatfield\-Dodds, N\. DasSarma, E\. Tran\-Johnson,et al\.\(2022\)Language models \(mostly\) know what they know\.arXiv preprint arXiv:2207\.05221\.Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[21\]W\. Kim, Y\. In, S\. Park, D\. Lee, and C\. Park\(2026\)PAIR: prefix\-aware internal reward model for multi\-turn agent optimization\.arXiv preprint arXiv:2605\.17877\.Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[22\]T\. Kwiatkowski, J\. Palomaki, O\. Redfield, M\. Collins, A\. Parikh, C\. Alberti, D\. Epstein, I\. Polosukhin, J\. Devlin, K\. Lee,et al\.\(2019\)Natural questions: a benchmark for question answering research\.Transactions of the Association for Computational Linguistics7,pp\. 453–466\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p2.4),[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px2.p1.3)\.
- \[23\]Z\. Liang, Y\. Ma, B\. Chen, Z\. Qian, X\. Zhang, H\. Dai, and L\. Mao\(2026\)Search\-e1: self\-distillation drives self\-evolution in search\-augmented reasoning\.arXiv preprint arXiv:2605\.22511\.Cited by:[§2\.1](https://arxiv.org/html/2608.07531#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.107.105.105.15),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.262.260.260.15)\.
- \[24\]A\. Liu, Y\. Song, Z\. Chen, Z\. Gong, Z\. Yu, and J\. Wang\(2026\)ToolGate: token\-efficient pre\-call control for tool\-augmented vision\-language agents\.arXiv preprint arXiv:2606\.03054\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p3.1)\.
- \[25\]X\. Liu, X\. Yang, Z\. Li, P\. Li, and R\. He\(2026\)AgentHallu: benchmarking automated hallucination attribution of llm\-based agents\.arXiv preprint arXiv:2601\.06818\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p3.1),[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1),[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.220.218.218.15.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.65.63.63.15.1),[§7\.2\.1](https://arxiv.org/html/2608.07531#S7.SS2.SSS1.Px11.p1.1)\.
- \[26\]S\. Ma, C\. Deng, J\. Mao, J\. Huang, T\. Wang, J\. Wu, C\. Zhang,et al\.\(2025\)PoU: proof\-of\-use to counter tool\-call hacking in deepresearch agents\.arXiv preprint arXiv:2510\.10931\.Cited by:[§2\.1](https://arxiv.org/html/2608.07531#S2.SS1.p1.1)\.
- \[27\]A\. Mallen, A\. Asai, V\. Zhong, R\. Das, D\. Khashabi, and H\. Hajishirzi\(2023\)When not to trust language models: investigating effectiveness of parametric and non\-parametric memories\.InProceedings of the 61st annual meeting of the association for computational linguistics \(volume 1: Long papers\),pp\. 9802–9822\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p3.1)\.
- \[28\]C\. Pang, X\. Feng, Y\. Yi, Z\. Chen, J\. Hong, T\. Yao, N\. Yuan, J\. Luo, L\. Lu, and X\. Lou\(2026\)ICA: information\-aware credit assignment for visually grounded long\-horizon information\-seeking agents\.arXiv preprint arXiv:2602\.10863\.Cited by:[§2\.1](https://arxiv.org/html/2608.07531#S2.SS1.p1.1)\.
- \[29\]J\. Pang, R\. Cheng, Z\. Ye, X\. Ma, Z\. Wu, X\. Huang, and Y\. Jiang\(2026\)Steering the verifiability of multimodal ai hallucinations\.arXiv preprint arXiv:2604\.06714\.Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[30\]F\. Petroni, T\. Rocktäschel, S\. Riedel, P\. Lewis, A\. Bakhtin, Y\. Wu, and A\. Miller\(2019\)Language models as knowledge bases?\.InProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing \(EMNLP\-IJCNLP\),pp\. 2463–2473\.Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[31\]J\. Qi, G\. Sarti, R\. Fernández, and A\. Bisazza\(2024\-11\)Model internals\-based answer attribution for trustworthy retrieval\-augmented generation\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 6037–6053\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.347/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.347)Cited by:[§2\.1](https://arxiv.org/html/2608.07531#S2.SS1.p1.1)\.
- \[32\]Y\. Shang, Y\. Li, K\. Zhao, L\. Ma, J\. Liu, F\. Xu, and Y\. Li\(2025\)AgentSquare: automatic llm agent search in modular design space\.InInternational Conference on Learning Representations,Y\. Yue, A\. Garg, N\. Peng, F\. Sha, and R\. Yu \(Eds\.\),Vol\.2025,pp\. 3841–3865\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/file/0ae94013da7cd459402fd77874e09ee3-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p1.1)\.
- \[33\]A\. Singh, A\. Fry, A\. Perelman, A\. Tart, A\. Ganesh, A\. El\-Kishky, A\. McLaughlin, A\. Low, A\. Ostrow, A\. Ananthram,et al\.\(2025\)Openai gpt\-5 system card\.arXiv preprint arXiv:2601\.03267\.Cited by:[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.220.218.218.15.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.65.63.63.15.1),[§7\.2\.1](https://arxiv.org/html/2608.07531#S7.SS2.SSS1.Px11.p1.1)\.
- \[34\]H\. Song, J\. Jiang, Y\. Min, J\. Chen, Z\. Chen, W\. X\. Zhao, L\. Fang, and J\. Wen\(2025\)R1\-searcher: incentivizing the search capability in llms via reinforcement learning\.arXiv preprint arXiv:2503\.05592\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p3.1)\.
- \[35\]Z\. Sun, X\. Zang, K\. Zheng, J\. Xu, X\. Zhang, W\. Yu, Y\. Song, and H\. Li\(2025\)ReDeEP: detecting hallucination in retrieval\-augmented generation via mechanistic interpretability\.InInternational Conference on Learning Representations,Y\. Yue, A\. Garg, N\. Peng, F\. Sha, and R\. Yu \(Eds\.\),Vol\.2025,pp\. 50250–50279\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/file/7daf60e805e596c3bd1e843e72ea5560-Paper-Conference.pdf)Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[36\]Y\. Tang, C\. Yang, S\. Liu, Z\. Xiang, Z\. Chen, Q\. Zhang, and J\. Su\(2026\)SAAS: self\-aware reinforcement learning for over\-search mitigation in agentic search\.arXiv preprint arXiv:2605\.29796\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.07531#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.121.119.119.15.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.276.274.274.15.1)\.
- \[37\]Q\. Team\(2024\-09\)Qwen2\.5: a party of foundation models\.External Links:[Link](https://qwenlm.github.io/blog/qwen2.5/)Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p2.4),[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px1.p1.1)\.
- \[38\]H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. Sabharwal\(2022\-05\)MuSiQue: multihop questions via single\-hop question composition\.Transactions of the Association for Computational Linguistics10,pp\. 539–554\.External Links:ISSN 2307\-387X,[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00475),[Link](https://doi.org/10.1162/tacl_a_00475),https://direct\.mit\.edu/tacl/article\-pdf/doi/10\.1162/tacl\_a\_00475/2020694/tacl\_a\_00475\.pdfCited by:[§1](https://arxiv.org/html/2608.07531#S1.p2.4),[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px2.p1.3)\.
- \[39\]J\. Tuyls, D\. J\. Foster, A\. Krishnamurthy, and J\. T\. Ash\(2025\)Representation\-based exploration for language models: from test\-time to post\-training\.arXiv preprint arXiv:2510\.11686\.Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[40\]C\. Wang, X\. Liu, Y\. Yue, X\. Tang, T\. Zhang, C\. Jiayang, Y\. Yao, W\. Gao, X\. Hu, Z\. Qi,et al\.\(2023\)Survey on factuality in large language models: knowledge, retrieval and domain\-specificity\.arXiv preprint arXiv:2310\.07521\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p3.1)\.
- \[41\]G\. Wang, S\. Dai, G\. Ye, Z\. Gan, W\. Yao, Y\. Deng, X\. Wu, and Z\. Ying\(2026\)Information gain\-based policy optimization: a simple and effective approach for multi\-turn search agents\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.07531#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.248.246.246.15.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.93.91.91.15.1)\.
- \[42\]H\. Wang, C\. T\. Leong, J\. Wang, J\. Wang, and W\. Li\(2025\)Spa\-rl: reinforcing llm agents via stepwise progress attribution\.arXiv preprint arXiv:2505\.20732\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p3.1)\.
- \[43\]P\. Wu, M\. Zhang, X\. Zhang, X\. Du, and Z\. Chen\(2025\)Search wisely: mitigating sub\-optimal agentic searches by reducing uncertainty\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 19734–19745\.Cited by:[§2\.1](https://arxiv.org/html/2608.07531#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.122.120.120.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.277.275.275.1)\.
- \[44\]Z\. Xi, C\. Liao, G\. Li, Z\. Zhang, W\. Chen, B\. Wang, S\. Jin, Y\. Zhou, J\. Guan, W\. Wu,et al\.\(2026\)Agentprm: process reward models for llm agents via step\-wise promise and progress\.InProceedings of the ACM Web Conference 2026,pp\. 4184–4195\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.07531#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.206.204.204.15),[Table 1](https://arxiv.org/html/2608.07531#S4.T1.51.49.49.15)\.
- \[45\]Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. Cohen, R\. Salakhutdinov, and C\. D\. Manning\(2018\)HotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 conference on empirical methods in natural language processing,pp\. 2369–2380\.Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p2.4),[§4\.1](https://arxiv.org/html/2608.07531#S4.SS1.SSS0.Px2.p1.3)\.
- \[46\]Q\. Zhan, A\. Budiman\-Chan, A\. Zayed, X\. Guo, D\. Kang, and J\. Kim\(2026\-03\)SafeSearch: do not trade safety for utility in LLM search agents\.InFindings of the Association for Computational Linguistics: EACL 2026,V\. Demberg, K\. Inui, and L\. Marquez \(Eds\.\),Rabat, Morocco,pp\. 2800–2815\.External Links:[Link](https://aclanthology.org/2026.findings-eacl.146/),[Document](https://dx.doi.org/10.18653/v1/2026.findings-eacl.146),ISBN 979\-8\-89176\-386\-9Cited by:[§1](https://arxiv.org/html/2608.07531#S1.p1.1)\.
- \[47\]Y\. Zhang, H\. Huang, Z\. Song, Z\. Zhao, Q\. Zhang, Y\. Zhu, and D\. Zhao\(2026\-07\)CriticSearch: fine\-grained credit assignment for search agents via a retrospective critic\.InFindings of the Association for Computational Linguistics: ACL 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 12272–12290\.External Links:[Link](https://aclanthology.org/2026.findings-acl.596/),[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.596),ISBN 979\-8\-89176\-395\-1Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[48\]Y\. Zhang, J\. Chen, J\. Wang, Y\. Liu, C\. Yang, C\. Shi, X\. Zhu, Z\. Lin, H\. Wan, Y\. Yang,et al\.\(2024\)Toolbehonest: a multi\-level hallucination diagnostic benchmark for tool\-augmented large language models\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 11388–11422\.Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[49\]J\. Zhao, Y\. Yang, X\. Hu, J\. Tong, Y\. Lu, W\. Wu, T\. Gui, Q\. Zhang, and X\. Huang\(2026\)Understanding parametric and contextual knowledge reconciliation within large language models\.Advances in Neural Information Processing Systems38,pp\. 102978–103012\.Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.
- \[50\]L\. Zhao, S\. Zalouk, C\. K\. Belardi, J\. Lovelace, J\. P\. Zhou, R\. T\. Noonan, D\. Go, K\. Q\. Weinberger, Y\. Artzi, and J\. J\. Sun\(2025\)Pre\-training limited memory language models with internal and external knowledge\.arXiv preprint arXiv:2505\.15962\.Cited by:[§2\.2](https://arxiv.org/html/2608.07531#S2.SS2.p1.1)\.

## 7Appendix

### 7\.1Methodology supplements

##### Distributed implementation of the masked objective\.

For the masked average in[Equation˜10](https://arxiv.org/html/2608.07531#S3.E10), letℬs,w\\mathcal\{B\}\_\{s,w\}denote the rank\-local micro\-batch on data\-parallel workerwwat gradient\-accumulation stepss\. WithWWworkers andSSaccumulation steps, and assuming every local micro\-batch contains at least one selected token, the implementation\-level reduction is

⟨fi,j,t⟩ℳ,ℬ=1S​W​∑s=1S∑w=1W∑\(i,j\)∈ℬs,w∑tMi,j,t​fi,j,t∑\(i,j\)∈ℬs,w∑tMi,j,t\.\\left\\langle f\_\{i,j,t\}\\right\\rangle\_\{\\mathcal\{M\},\\mathcal\{B\}\}=\\frac\{1\}\{SW\}\\sum\_\{s=1\}^\{S\}\\sum\_\{w=1\}^\{W\}\\frac\{\\sum\_\{\(i,j\)\\in\\mathcal\{B\}\_\{s,w\}\}\\sum\_\{t\}M\_\{i,j,t\}f\_\{i,j,t\}\}\{\\sum\_\{\(i,j\)\\in\\mathcal\{B\}\_\{s,w\}\}\\sum\_\{t\}M\_\{i,j,t\}\}\.\(12\)Thus selected tokens are averaged within each rank\-local micro\-batch before the local losses are averaged across workers and accumulation steps\.

##### Clipped sampled\-token KL\.

For a generated tokenwi,j,tw\_\{i,j,t\}, the implemented low\-variance penalty is

ui,j,t\\displaystyle u\_\{i,j,t\}=log⁡πref​\(wi,j,t∣τi,j,<t\)−log⁡πθ​\(wi,j,t∣τi,j,<t\),\\displaystyle=\\log\\pi\_\{\\mathrm\{ref\}\}\(w\_\{i,j,t\}\\mid\\tau\_\{i,j,<t\}\)\-\\log\\pi\_\{\\theta\}\(w\_\{i,j,t\}\\mid\\tau\_\{i,j,<t\}\),\(13\)KL^i,j,tLV\\displaystyle\\widehat\{\\mathrm\{KL\}\}^\{\\mathrm\{LV\}\}\_\{i,j,t\}=clip⁡\(exp⁡\(ui,j,t\)−ui,j,t−1,−κ,κ\)\.\\displaystyle=\\operatorname\{clip\}\\\!\\left\(\\exp\(u\_\{i,j,t\}\)\-u\_\{i,j,t\}\-1,\-\\kappa,\\kappa\\right\)\.The fixed clipping bound limits outlier token penalties and, when active, truncates their gradients\.

##### Necessity\-factorization definitions\.

The searched branch usesni​di,jn\_\{i\}d\_\{i,j\}and the direct branch usesbib\_\{i\}\. The*w/o searched gate*variant replacesni​di,jn\_\{i\}d\_\{i,j\}bydi,jd\_\{i,j\};*w/o direct bonus*removesbib\_\{i\}; and*w/o both necessity terms*applies both changes\. The shuffled control replaces onlynin\_\{i\}by a cross\-question permutationn~i\\tilde\{n\}\_\{i\}, redrawn per seed, while keepingbib\_\{i\}paired with its original question\. Thus the shuffle tests searched\-branch assignment rather than the complete necessity signal or a uniform increase in reward magnitude; outcomes are reported in[Table˜4](https://arxiv.org/html/2608.07531#S4.T4)\. Notably, on Qwen2\.5\-3B the shuffled control \(0\.341/0\.869/3\.2040\.341/0\.869/3\.204\) lands close to*w/o both necessity terms*\(0\.339/0\.861/3\.2860\.339/0\.861/3\.286\) and well below the full method \(0\.368/0\.972/2\.8120\.368/0\.972/2\.812\), indicating that the gain comes from the correct within\-question pairing ofnin\_\{i\}anddi,jd\_\{i,j\}rather than merely from the presence of a necessity term\.

##### Proof and boundary cases for[Proposition˜1](https://arxiv.org/html/2608.07531#Thmproposition1)\.

We expand the controlled\-gate construction used in the main text\. For a fixed questionii,γ∈\[0,1\]\\gamma\\in\[0,1\]replacesnin\_\{i\}only in the correct\-searched reliance term, whilebib\_\{i\}and all other reward branches remain fixed\. For completeness, define

xi,j\\displaystyle x\_\{i,j\}=\{di,j,vi,j=1,ei,j=1,Ni,j\>0,0,otherwise,\\displaystyle=\(14\)𝐫i​\(γ\)\\displaystyle\\mathbf\{r\}\_\{i\}\(\\gamma\)=𝐚i\+λg​γ​𝐱i,𝐚i:=𝐫i​\(0\)\.\\displaystyle=\\mathbf\{a\}\_\{i\}\+\\lambda\_\{g\}\\gamma\\mathbf\{x\}\_\{i\},\\qquad\\mathbf\{a\}\_\{i\}=\\mathbf\{r\}\_\{i\}\(0\)\.The choicesγ=ni\\gamma=n\_\{i\}andγ=1\\gamma=1recover Search\-G1 and the controlled no\-gate comparison, respectively\. The latter changes only the searched reward coefficient; it does not redefinebib\_\{i\}\.

Let

H=IG−1G​𝟏𝟏⊤,𝐮i=H​𝐚i,𝐯i=H​𝐱i,H=I\_\{G\}\-\\frac\{1\}\{G\}\\mathbf\{1\}\\mathbf\{1\}^\{\\top\},\\qquad\\mathbf\{u\}\_\{i\}=H\\mathbf\{a\}\_\{i\},\\qquad\\mathbf\{v\}\_\{i\}=H\\mathbf\{x\}\_\{i\},and write𝐳i​\(γ\)=𝐮i\+λg​γ​𝐯i\\mathbf\{z\}\_\{i\}\(\\gamma\)=\\mathbf\{u\}\_\{i\}\+\\lambda\_\{g\}\\gamma\\mathbf\{v\}\_\{i\}\. Sinceσi​\(γ\)\\sigma\_\{i\}\(\\gamma\)is the sample standard deviation,

σi​\(γ\)=‖𝐳i​\(γ\)‖2G−1\.\\sigma\_\{i\}\(\\gamma\)=\\frac\{\\\|\\mathbf\{z\}\_\{i\}\(\\gamma\)\\\|\_\{2\}\}\{\\sqrt\{G\-1\}\}\.
###### Proof of[Proposition˜1](https://arxiv.org/html/2608.07531#Thmproposition1)\.

Centering[Equation˜14](https://arxiv.org/html/2608.07531#S7.E14)givesH​𝐫i​\(γ\)=𝐮i\+λg​γ​𝐯i=𝐳i​\(γ\)H\\mathbf\{r\}\_\{i\}\(\\gamma\)=\\mathbf\{u\}\_\{i\}\+\\lambda\_\{g\}\\gamma\\mathbf\{v\}\_\{i\}=\\mathbf\{z\}\_\{i\}\(\\gamma\)\. Substituting the sample standard deviation gives[Equation˜11](https://arxiv.org/html/2608.07531#S3.E11); itsjjth component is the scalar advantage in[Equation˜8](https://arxiv.org/html/2608.07531#S3.E8)\. Forδ=0\\delta=0,

𝐀i0​\(γ\)=G−1​𝐳i​\(γ\)‖𝐳i​\(γ\)‖2\.\\mathbf\{A\}\_\{i\}^\{0\}\(\\gamma\)=\\sqrt\{G\-1\}\\,\\frac\{\\mathbf\{z\}\_\{i\}\(\\gamma\)\}\{\\\|\\mathbf\{z\}\_\{i\}\(\\gamma\)\\\|\_\{2\}\}\.\(15\)Thus normalization removes centered reward scale and retains only direction\. Suppose𝐮i\\mathbf\{u\}\_\{i\}and𝐯i\\mathbf\{v\}\_\{i\}are linearly independent but the centered vectors for distinctγ1\\gamma\_\{1\}andγ2\\gamma\_\{2\}have the same direction\. Then for someq\>0q\>0,

𝐮i\+λg​γ1​𝐯i=q​\(𝐮i\+λg​γ2​𝐯i\)\.\\mathbf\{u\}\_\{i\}\+\\lambda\_\{g\}\\gamma\_\{1\}\\mathbf\{v\}\_\{i\}=q\(\\mathbf\{u\}\_\{i\}\+\\lambda\_\{g\}\\gamma\_\{2\}\\mathbf\{v\}\_\{i\}\)\.Linear independence forcesq=1q=1and thenγ1=γ2\\gamma\_\{1\}=\\gamma\_\{2\}, a contradiction\. If𝐮i\\mathbf\{u\}\_\{i\}and𝐯i\\mathbf\{v\}\_\{i\}are collinear, all nonzero𝐳i​\(γ\)\\mathbf\{z\}\_\{i\}\(\\gamma\)are collinear, and any global reversal between two gate values must pass through𝐳i​\(γ\)=𝟎\\mathbf\{z\}\_\{i\}\(\\gamma\)=\\mathbf\{0\}\. The denominator in[Equation˜11](https://arxiv.org/html/2608.07531#S3.E11)is positive forδ\>0\\delta\>0, so it changes magnitude but not direction\.

For the strict homogeneous case, suppose every trajectory is valid, correct, searched, and has the same search costcic\_\{i\}\. Then𝐚i=\(1−η​ci\)​𝟏\\mathbf\{a\}\_\{i\}=\(1\-\\eta c\_\{i\}\)\\mathbf\{1\}, so𝐮i=𝟎\\mathbf\{u\}\_\{i\}=\\mathbf\{0\}and𝐯i=H​𝐝i\\mathbf\{v\}\_\{i\}=H\\mathbf\{d\}\_\{i\}\. Letsd,is\_\{d,i\}be the sample standard deviation of\{di,j\}j=1G\\\{d\_\{i,j\}\\\}\_\{j=1\}^\{G\}\. Equation[Equation˜11](https://arxiv.org/html/2608.07531#S3.E11)reduces componentwise to

Ai,jδ​\(γ\)=λg​γ​\(di,j−d¯i\)λg​γ​sd,i\+δ\.A\_\{i,j\}^\{\\delta\}\(\\gamma\)=\\frac\{\\lambda\_\{g\}\\gamma\(d\_\{i,j\}\-\\bar\{d\}\_\{i\}\)\}\{\\lambda\_\{g\}\\gamma s\_\{d,i\}\+\\delta\}\.\(16\)Forγ\>0\\gamma\>0andsd,i\>0s\_\{d,i\}\>0,

𝐀i0​\(γ\)=H​𝐝isd,i,𝐀iδ​\(γ\)=λg​γ​sd,iλg​γ​sd,i\+δ​𝐀i0​\(γ\)\.\\mathbf\{A\}\_\{i\}^\{0\}\(\\gamma\)=\\frac\{H\\mathbf\{d\}\_\{i\}\}\{s\_\{d,i\}\},\\qquad\\mathbf\{A\}\_\{i\}^\{\\delta\}\(\\gamma\)=\\frac\{\\lambda\_\{g\}\\gamma s\_\{d,i\}\}\{\\lambda\_\{g\}\\gamma s\_\{d,i\}\+\\delta\}\\mathbf\{A\}\_\{i\}^\{0\}\(\\gamma\)\.\(17\)Hence every positive gate preserves rollout ranking, advantage signs, and credit direction; it cancels from the advantage values whenδ=0\\delta=0and only rescales the entire vector whenδ\>0\\delta\>0\. ∎

##### Boundary cases\.

Ifγ=0\\gamma=0, the stabilized advantage in the strict homogeneous case is𝐀iδ​\(0\)=𝟎\\mathbf\{A\}\_\{i\}^\{\\delta\}\(0\)=\\mathbf\{0\}, whereas the idealizedδ=0\\delta=0expression is0/00/0and undefined\. Ifsd,i=0s\_\{d,i\}=0, reliance provides no within\-group tie\-breaking; ifλg=0\\lambda\_\{g\}=0, the gate has no effect by construction\. A zero vector𝐳i​\(γ\)\\mathbf\{z\}\_\{i\}\(\\gamma\)is likewise degenerate\.

##### Mixed groups\.

Outcome branches, direct\-versus\-search branches, and unequal search costs can make𝐮i≠𝟎\\mathbf\{u\}\_\{i\}\\neq\\mathbf\{0\}\. In this case the gate can change the relative weight of𝐮i\\mathbf\{u\}\_\{i\}and𝐯i\\mathbf\{v\}\_\{i\}in[Equation˜11](https://arxiv.org/html/2608.07531#S3.E11)\. It changes the normalized credit direction for distinct positive gate values whenever these centered components are non\-collinear; heterogeneous composition alone is not sufficient\. For any two nonzero comparison vectors, the cosine diagnostic is exactly the cosine between𝐳i​\(γ1\)\\mathbf\{z\}\_\{i\}\(\\gamma\_\{1\}\)and𝐳i​\(γ2\)\\mathbf\{z\}\_\{i\}\(\\gamma\_\{2\}\), because the stabilizer changes only their positive scalar magnitudes\. A component changes sign exactly when

zi,j​\(γ1\)​zi,j​\(γ2\)<0\.z\_\{i,j\}\(\\gamma\_\{1\}\)z\_\{i,j\}\(\\gamma\_\{2\}\)<0\.
Cosine similarity is undefined and sign\-flip statistics are convention\-dependent for zero comparison vectors\. We therefore rely on the analytic characterization above; the controlled gate ablation in[Table˜4](https://arxiv.org/html/2608.07531#S4.T4)provides complementary downstream evidence\.

### 7\.2Experimental setup and reproduction

#### 7\.2\.1Implementation details

##### Data and retrieval\.

Unless stated otherwise, methods share policy\-training data, the retriever, and the global interaction/update cap\. Benchmark examples come fromRUC\-NLPIR/FlashRAG\_datasetsusing the benchmark\-native training and available held\-out evaluation splits\. Retrieval uses the Wikipedia 2018 corpus \(wiki\-18\.jsonl\),intfloat/e5\-base\-v2mean\-pooled embeddings, and a FAISS Flat inner\-product index, returning three passages per executed retrieval action\.

##### Rollout and optimization\.

We allow four interaction rounds and maximum prompt/response/start/observation lengths of 4096/500/2048/500\. Fixed\-rollout methods use group size five; adaptive methods reallocate samples within the same cap\. GRPO uses sample\-standard\-deviation normalization withδ=10−6\\delta=10^\{\-6\}, temperature 1\.0, top\-p=0\.95p=0\.95, top\-k=−1k=\-1, batch size 128, learning rate10−610^\{\-6\}, and warmup ratio 0\.285\.

##### Policy update and evaluation\.

The clipped update uses one epoch,ϵ=0\.2\\epsilon=0\.2, mini\-/micro\-batches of 32/8, entropy coefficientαH=0\.001\\alpha\_\{\\mathrm\{H\}\}=0\.001, and the sampled\-token KL term in[Equation˜13](https://arxiv.org/html/2608.07531#S7.E13)withβKL=0\.001\\beta\_\{\\mathrm\{KL\}\}=0\.001and fixedκ=10\\kappa=10\. Prompt, observation, and padding tokens are excluded from the actor loss\. Unless a table specifies sampling or a fixed subset, single\-trajectory metrics use greedy decoding on the full benchmark\-specific held\-out split\. Evaluation takes an explicit checkpoint path and performs no automatic checkpoint search\.

##### Data separation\.

Policy\-training questions, the 300\-question readout\-calibration pool, architecture\-selection groups, and benchmark evaluation questions are disjoint and identified by stable item IDs\. Trained methods use four independently seeded policies, whereas Base and Prompted Search are untrained point estimates\.

##### Statistical unit\.

The independent training run is the unit of replication for trained\-method comparisons\. With four runs, a two\-sided 95%ttinterval has half\-width approximately1\.59​SD1\.59\\,\\mathrm\{SD\}, so small mean differences cannot support superiority by mean ordering alone\. Using the seed\-by\-question matrices, we propagate both sources of variation\. Accordingly,[Table˜2](https://arxiv.org/html/2608.07531#S4.T2)reports a completed hierarchical significance analysis: it resamples training seeds and, within each sampled seed, paired questions, then applies Holm correction across EM, TC, D\-Inv, and Search/Q within each comparator–scale family\. Under this analysis the TC difference is significant at both scales, and the D\-Inv and Search/Q differences at 7B; EM against the strongest baseline remains competitive rather than statistically superior\.

##### Reward and parser settings\.

We setλg=0\.6\\lambda\_\{g\}=0\.6,η=0\.02\\eta=0\.02,αcb=0\.5\\alpha\_\{\\mathrm\{cb\}\}=0\.5,rinv=−0\.1r\_\{\\mathrm\{inv\}\}=\-0\.1, andrwrong=0\.05r\_\{\\mathrm\{wrong\}\}=0\.05, satisfyingrinv<rwrong<1−ηr\_\{\\mathrm\{inv\}\}<r\_\{\\mathrm\{wrong\}\}<1\-\\eta\. The default\(λg,η\)=\(0\.6,0\.02\)\(\\lambda\_\{g\},\\eta\)=\(0\.6,0\.02\)is fixed from the group\-split calibration/validation data and the sweeps in[Tables˜A13](https://arxiv.org/html/2608.07531#S7.T13)and[A14](https://arxiv.org/html/2608.07531#S7.T14), not selected on the test splits; baseline\-specific coefficients are tuned under the same validation protocol and shared budget\. The search cost follows[Equation˜5](https://arxiv.org/html/2608.07531#S3.E5)and is computed from the decoded policy\-generated response only:ci,j=𝟏​\[Ni,j\>1\]=min⁡\{1,\(Ni,j−1\)\+\}c\_\{i,j\}=\\mathbf\{1\}\[N\_\{i,j\}\>1\]=\\min\\\{1,\(N\_\{i,j\}\-1\)\_\{\+\}\\\}\. Markers are counted on the policy\-generated response only, so in\-context demonstrations in the prompt are never counted\. The released run manifest fixes this response\-only parser and ships unit tests for direct, one\-search, repeated\-search, invalid\-action, wrong\-answer, and correct\-answer trajectories\.

##### Evaluation\-time resource accounting\.

The evaluator records response\-side marker count, policy\-generated tokens, retrieved\-observation tokens, and wall\-clock time for every question\. Policy\-generated tokens are obtained from the response\-sideinfo\_mask; observation tokens are the response\-side attention\-mask count minus that quantity\. These per\-question token columns are reported in[Tables˜A4](https://arxiv.org/html/2608.07531#S7.T4)and[A5](https://arxiv.org/html/2608.07531#S7.T5)\.

##### Readout refresh and overhead\.

Readouts are selected from logistic regression and MLPs with hidden sizes 64, 128, and\(256,128\)\(256,128\)on frozen states\. At each refit, architecture and layer\{21,24,27\}\\\{21,24,27\\\}are selected by group\-split validation AUC separately forDϕD\_\{\\phi\}andBψB\_\{\\psi\}\. The nominal schedule emits checkpoints everyK=25K=25updates and refits both readouts asynchronously onM=300M=300calibration questions\. A new evaluator is installed atomically only after both readouts finish; otherwise the previous complete evaluator remains active, so the realized refresh interval may exceed the nominal cadence\. Sensitivity appears in[Table˜A14](https://arxiv.org/html/2608.07531#S7.T14); logistic candidates use class balancing and MLPs use early stopping withL2=10−4L\_\{2\}=10^\{\-4\}\.

Each refit uses one no\-retrieval decode per calibration question and one evidence\-deleted decode per valid searched calibration rollout\. Its extra decode count is thereforeM\+Nvs≤M​\(1\+G\)M\+N\_\{\\mathrm\{vs\}\}\\leq M\(1\+G\): atM=300M=300andG=5G=5, at most1\.81\.8k decodes, or approximately1111–12%12\\%of the16,00016\{,\}000rollout decodes in a nominalK=25K=25cycle\. Reward\-time scoring additionally requires a batched, no\-gradient teacher\-forced pass through the frozen snapshot, but no LLM\-judge call during policy updates\.

##### Baseline\-specific operations\.

The common on\-policy phase uses the same policy\-training data, base model, retrieval environment, trajectory format, and global interaction/update cap\. Adaptive methods may reallocate rollouts within that cap, while each baseline retains its defining auxiliary computation; the comparison therefore matches shared on\-policy exposure rather than end\-to\-end compute\.

The Search\-R1 baseline uses terminal correctness and a fixed0\.50\.5correct\-direct bonus, but no necessity or reliance readout\. Matching the maximum direct\-answer bonus avoids comparison only against a bonus\-free objective, but does not make the rewards identical: Search\-G1 uses the state\-dependent bonus0\.5​bi0\.5b\_\{i\}, necessity\-conditioned reliance credit, and repeated\-search cost\. We therefore treat Search\-R1 as an outcome\-plus\-directness reference, not an isolated causal estimate of the readouts’ effect\.

##### Per\-baseline reproduction audit\.

[Table˜A1](https://arxiv.org/html/2608.07531#S7.T1)records whether each implementation follows the original algorithm directly \(F\) or adapts it to the shared environment \(A\), together with the retained auxiliary computation and principal adaptation\.

##### External\-judge baseline citation\.

TheGPT\-5 Judgebaseline applies the evidence\-support scoring protocol adapted fromLiuet al\.\[[25](https://arxiv.org/html/2608.07531#bib.bib6)\]with the judge model described bySinghet al\.\[[33](https://arxiv.org/html/2608.07531#bib.bib45)\]\. The judge version and prompt are fixed across runs\.

Table A1:Baseline implementation audit\. “F” denotes an algorithm\-faithful implementation under the shared protocol and “A” an adapted port\.MethodRepro\.Auxiliary computeMain deviation from officialSearch\-R1Anonedirect\-bonus scale matchedGiGPOFnoneanchor\-group size set to shared capAgentPRMAfixed process\-reward modelPRM checkpoint/prompt fixedGPT\-5 JudgeAfixed external judgejudge prompt/model fixedARPOFnonerollout reallocation within shared capIGPOFshared\-model likelihood passesconfidence estimator from shared modelSearch\-E1Foffline self\-distillationsibling\-context buffer within capSAASAdisabled/enabled rollout groupsrollout allocation and schedule aligned to our capβ\\beta\-GRPOAquery\-token log probabilitiesconfidence gate retained under the shared protocolKbPOAparametric samples \+ quality judgequadrant reward aligned to the shared reward range

#### 7\.2\.2Artifact release checklist

All reported results are produced by the released scripts\.[Table˜A2](https://arxiv.org/html/2608.07531#S7.T2)lists, for each result family, the artifact provided for independent reproduction; the full bundle—training and evaluation code, per\-run predictions, and aggregation scripts—accompanies the submission\.

Table A2:Artifact manifest\. Each result family is reproducible from the released scripts; rows list the artifacts provided for independent reproduction\.Result familyRetained evidenceProvided reproduction artifactFull\-held\-out Base boundary countsdataset, model, count summariesitem ID, prediction, exact/inclusion labels2Wiki necessity\-score refitsmatched 300\-item dumps and readoutsitem ID, hard label, score, checkpoint, split hash3B deletion\-label robustnesspaired decoded outputs and item IDspaired decoded outputs and item IDsMain comparisonfull predictions and run summariesmethod, seed, checkpoint, full predictions, aggregation scriptBar and line figures \(3B/7B\)final plots and aggregate arraysplotting arrays, item splits, and generation scriptSecondary ablations and sensitivity tablesfour\-run records and evaluation outputsfour versioned runs and full evaluation outputsHierarchical significance analysisseed×\\timesitem paired predictions and endpointsseed, item ID, paired predictions, all four endpoint valuesGate and credit\-usefulness diagnosticsreplay arrays and support annotationsgroup IDs, branch rewards,dd,nn, advantages, support labelsHuman grounding and common\-question analysesitem\-level annotations and adjudicationsitem IDs, blinded outputs, raters, judgments, adjudicationShortcut controls and baseline tuning ledgermatched probe outputs and per\-baseline sweepsmatched probe outputs and per\-baseline validation logsToken costper\-question NQ cost auditper\-question markers, generated/observation tokens, latency

For every trained\-method row, the release manifest maps

method→model→dataset→seed→checkpoint hash→prediction file hash→aggregation commit\.\\displaystyle\\text\{method\}\\rightarrow\\text\{model\}\\rightarrow\\text\{dataset\}\\rightarrow\\text\{seed\}\\rightarrow\\text\{checkpoint hash\}\\rightarrow\\text\{prediction file hash\}\\rightarrow\\text\{aggregation commit\}\.

The same item IDs should be used for paired task, grounding, search, and token metrics\.

### 7\.3Results analysis and additional diagnostics

#### 7\.3\.1Evaluation definitions and diagnostics

Table A3:Full held\-out question and closed\-book boundary composition for Base Qwen2\.5\-3B\-Instruct\. Cells report count \(percentage\)\. Structural labels are dataset\-provided and are not interchangeable with policy\-relative retrieval necessity\.DatasetNNExact CB\-correctExact CB\-wrongInclusion sufficientInclusion necessaryDataset structureNQ3,610512 \(14\.18%\)3,098 \(85\.82%\)761 \(21\.08%\)2,849 \(78\.92%\)single\-hop benchmark; no item\-level hop fieldHotpotQA7,4051,134 \(15\.31%\)6,271 \(84\.69%\)1,437 \(19\.41%\)5,968 \(80\.59%\)79\.92% bridge; 20\.08% comparison2WikiMultiHopQA12,5762,665 \(21\.19%\)9,911 \(78\.81%\)3,115 \(24\.77%\)9,461 \(75\.23%\)41\.63% compositional; 24\.17% comparison; 12\.32% inference; 21\.88% bridge–comparisonMuSiQue2,41771 \(2\.94%\)2,346 \(97\.06%\)101 \(4\.18%\)2,316 \(95\.82%\)51\.80% 2\-hop; 31\.44% 3\-hop; 16\.76% 4\-hop

Table A4:Measured NQ cost audit on the held\-out split \(Qwen2\.5\-3B, greedy decoding, matched interaction budget\)\. Tokens are per question\. 7B follows the same protocol \([Table˜A5](https://arxiv.org/html/2608.07531#S7.T5)\)\.MethodEM↑\\uparrowSearch/QGen\. tok/QObs\. tok/QBase0\.1420\.000300Prompted Search0\.2502\.215120500Search\-R10\.3523\.281149510Search\-G10\.4532\.09484\.8468\.1

Table A5:Measured NQ cost audit for Qwen2\.5\-7B\-Instruct on the held\-out split \(greedy decoding, matched interaction budget\), companion to the 3B audit in[Table˜A4](https://arxiv.org/html/2608.07531#S7.T4)\. Tokens are per question\.MethodEM↑\\uparrowSearch/QGen\. tok/Q↓\\downarrowObs\. tok/Q↓\\downarrowBase0\.1920\.000280Prompted Search0\.3492\.171110490Search\-R10\.3933\.090140500Search\-G10\.5121\.86078440

##### Grounding metric definitions\.

TC’s searched\-trajectory component and D\-Inv use a document\-level counterfactual intervention on originally correct searched trajectories\. The evaluator replaces the gold answer in answer\-bearing retrieved documents with an alternative entity or value, then regenerates an answer from the modified documents\. Every evaluated correct searched trajectory is assigned exactly one class:Truewhen its normalized answer changes to track the injected counterfactual entity or value,Hallwhen it remains invariant and preserves the original correct answer, andAmbotherwise\. This evaluation protocol is distinct from the evidence\-deletion labels used to fit the evidence\-reliance readout: the training\-time label perturbs evidence by*deletion*and is used only to fitDϕ\(m\)D\_\{\\phi^\{\(m\)\}\}, while TC/D\-Inv perturb evidence by*document\-level entity replacement*and are computed post hoc without feeding back into the reward\.

##### Proxy scope and anti\-gaming triangulation\.

The deletion intervention retains the reasoning prefix and compares a sampled realized answer with a deterministic counterfactual decode\. It therefore measures answer\-stage sensitivity conditional on that prefix, rather than the total causal effect of retrieval; stricter prefix deletion and paired\-decoding audits appear in[Tables˜A6](https://arxiv.org/html/2608.07531#S7.T6)and[A16](https://arxiv.org/html/2608.07531#S7.T16)\. A learned readout can also be gamed\. Document replacement provides a non\-identical held\-out intervention with no shared gradient path, but the two tests are not statistically independent: for example, a strategy that copies document entities could respond to both deletion and replacement without genuine evidential grounding\. We therefore interpret concordant deletion\- and replacement\-based results as triangulation against a single simple proxy shortcut, not as a formal bound on gaming\.

Let𝒞\\mathcal\{C\}be the union of originally correct direct trajectories and originally correct searched trajectories classified by the intervention\.TCis the mean graded trust\-consistency score

TC=1\|𝒞\|​∑τ∈𝒞s​\(τ\),s​\(τ\)∈\{1\.0,0\.6,0\.3\}\.\\mathrm\{TC\}=\\frac\{1\}\{\|\\mathcal\{C\}\|\}\\sum\_\{\\tau\\in\\mathcal\{C\}\}s\(\\tau\),\\qquad s\(\\tau\)\\in\\\{1\.0,0\.6,0\.3\\\}\.Correct direct trajectories receive score1\.01\.0by convention; classified correct searched trajectories receive1\.01\.0forTrue,0\.60\.6forAmb, and0\.30\.3forHall\. Consequently, TC also varies with the direct/searched composition of the correct set; we interpret it jointly with D\-Inv, which is computed only on correct searched trajectories and is unaffected by this direct\-score convention\. Higher TC indicates a stronger operational grounding response\.D\-Invis counterfactual\-document invariance among classified originally correct searched trajectories,

D​\-​Inv=NHallNTrue\+NHall\+NAmb,\\mathrm\{D\\text\{\-\}Inv\}=\\frac\{N\_\{\\mathrm\{Hall\}\}\}\{N\_\{\\mathrm\{True\}\}\+N\_\{\\mathrm\{Hall\}\}\+N\_\{\\mathrm\{Amb\}\}\},so lower is better\. TC and D\-Inv are related but not redundant: TC preserves the graded ambiguous outcome and the full correct\-trajectory aggregate, whereas D\-Inv isolates invariance after the counterfactual intervention\. Together they provide the retained operational grounding evidence\. The human evidence\-support evaluation and common\-question outcome decomposition in the appendix provide complementary empirical corroboration \([Tables˜A10](https://arxiv.org/html/2608.07531#S7.T10)and[7\.3\.4](https://arxiv.org/html/2608.07531#S7.SS3.SSS4.Px5)\)\. Unless noted,[Table˜1](https://arxiv.org/html/2608.07531#S4.T1)reports TC and D\-Inv on NQ\.

##### Layer\-wise linear separability\.

We further examine where the deletion\-sensitivity label is most linearly separable\. For each layer, we train the same linear readout to distinguish deletion\-sensitive from deletion\-invariant correct trajectories and compare separability across depth\. The best displayed layer varies across settings—layer 27/21 on NQ and 24/21 on 2Wiki for 3B/7B—so the diagnostic supports selecting the layer per model and refit rather than fixing one universal depth \([Figure˜A1](https://arxiv.org/html/2608.07531#S7.F1)\)\.

![Refer to caption](https://arxiv.org/html/2608.07531v1/x4.png)Figure A1:Layer\-wise linear separability of the commit\-token deletion\-sensitivity label on NQ and 2WikiMultiHopQA using Qwen2\.5\-3B and Qwen2\.5\-7B\. Each point reports AUC among EM=1=1rollouts\.
##### Counterfactual label reliability\.

Search\-G1 operationalizes evidence reliance by whether the realized answer changes under evidence ablation \([Equation˜1](https://arxiv.org/html/2608.07531#S3.E1)\)\. The displayed 3B and 7B NQ/HotpotQA rows compare the default deletion label against a stricter intervention that also removes intermediate reasoning text\. Agreement is approximately 0\.95 withκ≈0\.88\\kappa\\approx 0\.88, consistent with limited reasoning\-prefix leakage in the audited settings but not proof of its absence elsewhere\.

Table A6:Counterfactual label reliability across model scales\.DatasetAgreement↑\\uparrowκ\\kappa↑\\uparrowReliance RateStrict Reliance RateQwen2\.5\-3B\-InstructNQ0\.9530\.8810\.7240\.747HotpotQA0\.9520\.8790\.7120\.741Qwen2\.5\-7B\-InstructNQ0\.9560\.8970\.6810\.704HotpotQA0\.9540\.8940\.6680\.696

##### Metric definitions for the closed\-book analyses\.

We define the metrics used in[Tables˜A7](https://arxiv.org/html/2608.07531#S7.T7)and[A8](https://arxiv.org/html/2608.07531#S7.T8)\. The HotpotQA known\-answer audit defines CB\-correct separately for each evaluated policy using its deterministic retrieval\-disabled decode; the 2Wiki audit instead fixes CB\-wrong using the corresponding Base model, so the latter supplies a common retrieval\-beneficial question set within each scale\.CB\-EMandCBw\-EMare exact\-match accuracies on the respective conditional sets\.CB\-S/QandAll\-S/Qare the mean numbers of response\-side parsed search markers per question on the CB\-correct set and on the full audited slice\.Over\-Sis the fraction of valid evaluated trajectories containing more than one parsed search marker\.Any@3is the fraction of evaluated CB\-wrong questions with at least one exact\-match\-correct sampled trajectory\. Metrics condition on trajectories or questions with a valid parsed answer; the valid counts may therefore differ across methods\. These behavioral metrics do not involve evidence perturbation\.

Table A7:Known\-answer preservation analysis on a 1,000\-question HotpotQA slice, with CB\-correct defined per policy from its retrieval\-disabled decode\.MethodCB\-EM↑\\uparrowCB\-S/Q↓\\downarrowAll\-S/Q↓\\downarrowOver\-S↓\\downarrowQwen2\.5\-3B\-InstructPrompted Search0\.3801\.1991\.2150\.225Search\-R10\.5411\.5351\.6290\.489Search\-G10\.5731\.0561\.0490\.040Qwen2\.5\-7B\-InstructPrompted Search0\.4311\.0851\.1020\.194Search\-R10\.5921\.4111\.5030\.438Search\-G10\.6410\.9140\.9050\.028

Table A8:Retrieval analysis on a fixed Base\-model CB\-wrong 2WikiMultiHopQA slice, evaluated on a common item\-level set within each scale\.MethodCBw\-EM↑\\uparrowAny@3↑\\uparrowS/Q↓\\downarrowOver\-S↓\\downarrowQwen2\.5\-3B\-InstructPrompted Search0\.1450\.2361\.1290\.315Search\-R10\.2540\.3331\.9720\.809Search\-G10\.3270\.4351\.6490\.605Qwen2\.5\-7B\-InstructPrompted Search0\.1620\.2521\.0500\.284Search\-R10\.2780\.3621\.8400\.748Search\-G10\.3560\.4761\.5200\.548

##### Search\-action frequency and known\-answer preservation\.

We test whether retrieval training preserves answers within each policy’s estimated knowledge boundary\. On the audited HotpotQA slice, Search\-G1 has the highest CB\-EM and the lowest CB\-S/Q and All\-S/Q among the compared methods\. Relative to Search\-R1, CB\-EM is 0\.573 versus 0\.541 at 3B and 0\.641 versus 0\.592 at 7B; All\-S/Q is 1\.049 versus 1\.629 and 0\.905 versus 1\.503 \([Table˜A7](https://arxiv.org/html/2608.07531#S7.T7)\)\. Because the CB\-correct set is method\-relative, these values are descriptive rather than common\-question effects\. The following analysis separately studies a fixed Base\-model CB\-wrong set on 2Wiki\.

##### Closed\-book\-wrong question performance\.

We next evaluate the CB\-wrong subset, on which the base model fails without retrieval\. On 2WikiMultiHopQA, Search\-G1 has higher reported mean EM than Prompted Search and the Search\-R1 baseline while using fewer mean search markers than the latter\. Relative to the Search\-R1 baseline, its mean EM is 0\.327 rather than 0\.254 at 3B and 0\.356 rather than 0\.278 at 7B; mean Search/Q is 1\.649 rather than 1\.972 at 3B and 1\.520 rather than 1\.840 at 7B \([Table˜A8](https://arxiv.org/html/2608.07531#S7.T8)\)\. Thus, in this subset, lower mean search\-marker frequency co\-occurs with higher mean EM\.

##### Cross\-dataset transfer\.

[Table˜A9](https://arxiv.org/html/2608.07531#S7.T9)descriptively maps distribution shift: each Search\-G1 policy is trained on one source and evaluated on the four benchmark\-specific held\-out splits\. Diagonal cells are in\-domain and off\-diagonal cells are cross\-dataset\. Across the two scales, 23 of the 24 off\-diagonal cells have mean EM above the fixed Prompted Search reference, although TC and Search/Q vary by source and target\. Because the full matrix contains no source\-matched RL baseline or seed uncertainty, it does not by itself isolate a method\-specific transfer gain; the NQ\-source comparison in[Table˜A23](https://arxiv.org/html/2608.07531#S7.T23)supplies the trained baseline available in this study\.

Table A9:In\-domain and cross\-dataset matrix\. Each cell reports EM / TC / Search/Q; diagonal cells are in\-domain and off\-diagonal cells are cross\-dataset\.Training DatasetTarget Test DatasetNQHotpotQA2WikiMuSiQueQwen2\.5\-3B\-InstructPrompted Search0\.250 / 0\.930 / 2\.2150\.199 / 0\.873/ 2\.5310\.238 / 0\.877 / 2\.9510\.057 / 0\.911 / 2\.141NQ0\.453 / 0\.976 / 2\.0940\.248 / 0\.919 / 2\.4800\.256 / 0\.917 / 2\.5510\.091 / 0\.990 / 2\.406HotpotQA0\.359 / 0\.967 / 2\.2050\.306 / 0\.911 / 2\.5490\.270 / 0\.909 / 2\.8280\.099 / 0\.936 / 2\.8812Wiki0\.377 / 0\.948 / 2\.4860\.316 / 0\.939 / 2\.4390\.368 / 0\.972 / 2\.8120\.116 / 0\.995 / 3\.275MuSiQue0\.297 / 0\.958 / 3\.6400\.241 / 0\.910 / 3\.3890\.279 / 0\.884 / 3\.2270\.143 / 0\.987 / 3\.443Qwen2\.5\-7B\-InstructPrompted Search0\.349 / 0\.936 / 2\.1710\.299 / 0\.883 / 2\.4800\.235 / 0\.887 / 2\.8920\.058 / 0\.918 / 2\.098NQ0\.512 / 0\.981 / 1\.8600\.365 / 0\.935 / 2\.2020\.330 / 0\.934 / 2\.2650\.162 / 0\.992 / 2\.137HotpotQA0\.406 / 0\.974 / 1\.9580\.450 / 0\.929 / 2\.2630\.349 / 0\.927 / 2\.5110\.176 / 0\.949 / 2\.5582Wiki0\.426 / 0\.958 / 2\.2080\.465 / 0\.951 / 2\.1660\.475 / 0\.978 / 2\.4950\.206 / 0\.996 / 2\.908MuSiQue0\.336 / 0\.966 / 3\.2320\.354 / 0\.928 / 3\.0090\.360 / 0\.907 / 2\.8660\.254 / 0\.990 / 3\.057

##### Independent human grounding evaluation\.

TC and D\-Inv are counterfactual\-document diagnostics, whereas the training\-time evidence\-reliance label uses evidence deletion\. We therefore conduct an intervention\-independent human evaluation on*correct searched*trajectories, where answer correctness is already fixed and the remaining question is whether the retrieved evidence supports the correct answer\. The following tables report the annotation design and the measured support values on these correct\-searched trajectories\.

Table A10:Descriptive human\-grounding on independent 180\-instance correct\-searched pools per method\. S@C is mean normalized support; Full and Uns are fully supported and unsupported fractions\.MethodS@C \(↑\\uparrow\)Full \(↑\\uparrow\)Uns\./CBU \(↓\\downarrow\)Prompted Search0\.5940\.4000\.211Search\-R10\.6500\.4720\.172GPT\-5 Judge0\.7970\.6780\.083Search\-E10\.7190\.5610\.122w/o Evidence\-Reliance Reward0\.6330\.4500\.183Search\-G1 \(ours\)0\.8420\.7330\.050

Confirmatory paired test \(Search\-G1 vs\. GPT\-5 Judge\)\.This confirmatory test uses270270matched pairs with shared question IDs and is computed independently of the descriptive six\-method rates in[Table˜A10](https://arxiv.org/html/2608.07531#S7.T10)\.[Table˜A11](https://arxiv.org/html/2608.07531#S7.T11)reports the per\-dataset paired effects; each pair contributes both method outputs, three pre\-adjudication ratings per output, and the paired bootstrap replicates\.

Table A11:Same\-question human\-support \(Search\-G1 minus GPT\-5 Judge\)\. Each dataset contributes distinct matched questions; cells report paired difference \[95% CI\]\. These paired effects are computed on the matched\-question pool and differ from the independent\-pool differences in[Table˜A10](https://arxiv.org/html/2608.07531#S7.T10)\.DatasetMatchedNNΔ\\DeltaS@C↑\\uparrowΔ\\DeltaFull↑\\uparrowΔ\\DeltaCBU↓\\downarrowNQ90\+0\.052​\[\+0\.017,\+0\.087\]\+0\.052\\,\[\+0\.017,\+0\.087\]\+0\.067​\[\+0\.022,\+0\.112\]\+0\.067\\,\[\+0\.022,\+0\.112\]−0\.033​\[−0\.064,−0\.002\]\-0\.033\\,\[\-0\.064,\-0\.002\]HotpotQA90\+0\.043​\[\+0\.009,\+0\.077\]\+0\.043\\,\[\+0\.009,\+0\.077\]\+0\.054​\[\+0\.011,\+0\.097\]\+0\.054\\,\[\+0\.011,\+0\.097\]−0\.044​\[−0\.077,−0\.011\]\-0\.044\\,\[\-0\.077,\-0\.011\]2Wiki90\+0\.050​\[\+0\.016,\+0\.084\]\+0\.050\\,\[\+0\.016,\+0\.084\]\+0\.059​\[\+0\.017,\+0\.101\]\+0\.059\\,\[\+0\.017,\+0\.101\]−0\.034​\[−0\.064,−0\.004\]\-0\.034\\,\[\-0\.064,\-0\.004\]All270\+0\.048​\[\+0\.026,\+0\.070\]\+0\.048\\,\[\+0\.026,\+0\.070\]\+0\.060​\[\+0\.032,\+0\.088\]\+0\.060\\,\[\+0\.032,\+0\.088\]−0\.037​\[−0\.057,−0\.017\]\-0\.037\\,\[\-0\.057,\-0\.017\]

Annotation\.Three annotators independently label each instance on a33\-point support scale: \(i\)Fully supported\(22\): the evidence explicitly states, or unambiguously entails through a short and complete reasoning chain, every fact required for the answer; \(ii\)Partially supported\(11\): the evidence supports only part of the answer, covers only some hops, or is relevant but insufficient to entail the answer; \(iii\)Unsupported\(0\): the evidence is irrelevant, contradicts the answer, or does not contain enough information to support the answer\. The final label is the majority vote; all\-disagreement cases are adjudicated by a fourth senior annotator\.

Target measures\.We reportSupport@Correct\(S@C\), the mean majority\-vote support score normalized to\[0,1\]\[0,1\];Fully Supported\(Full\), the fraction of score\-22instances; andUnsupported\(Uns\.\), the fraction of score\-0instances\. Since all evaluated trajectories are exact\-match correct, Uns\. is also the*Correct\-but\-Unsupported*\(CBU\) rate: the fraction of correct answers not supported by the retrieved evidence shown to annotators\. This evaluation uses no evidence\-deletion labels, counterfactual document replacement, re\-decoding, or Search\-G1 reward readouts\.

Agreement\.The annotation reaches substantial pre\-adjudication agreement \(Fleiss’κ=0\.79\\kappa=0\.79, ordinal Krippendorff’sα=0\.81\\alpha=0\.81on Qwen2\.5\-3B\-Instruct\)\.

[Table˜A10](https://arxiv.org/html/2608.07531#S7.T10)specifies a six\-method descriptive comparison in which each method contributes a distinct correct\-searched pool\. The same\-question table above \(Search\-G1 vs\. GPT\-5 Judge\) and the unconditional common\-question decomposition in[Section˜7\.3\.4](https://arxiv.org/html/2608.07531#S7.SS3.SSS4.Px5)provide the matched designs that address this conditioning\.

#### 7\.3\.2Sensitivity and controlled comparisons

##### Shared anchors and variant definitions\.

Several tables reuse the same four trained policies as comparison anchors rather than reporting additional repetitions \([Tables˜4](https://arxiv.org/html/2608.07531#S4.T4),[A13](https://arxiv.org/html/2608.07531#S7.T13),[A14](https://arxiv.org/html/2608.07531#S7.T14)and[A15](https://arxiv.org/html/2608.07531#S7.T15)\)\. Theη=0\\eta=0row is the*w/o search cost*condition,K=∞K=\\inftyis the frozen\-readout condition, and*w/o searched gate*is the main no\-gate condition\. In the latter,nin\_\{i\}is replaced by11only in the correct\-searched reliance term;bib\_\{i\}and all other branches remain unchanged\. The four\-round Search\-G1 entry is the default interaction\-budget anchor\.

##### Validation\-based hyperparameter selection\.

Search\-G1 coefficient and refresh sweeps and a baseline\-by\-baseline tuning ledger are recorded in the archive\.[Table˜A12](https://arxiv.org/html/2608.07531#S7.T12)discloses them for a fair comparison\. The same validation rule is used for every method: retain configurations within 0\.02 EM of the best validation EM, maximize TC, then minimize Search/Q; test data are never consulted\.

Table A12:Validation\-selection ledger\. Candidate counts and per\-method selections use the shared validation rule; Search\-G1’s displayed default matches the archived coefficient/update sweeps\. “Scale” is relative to the official auxiliary\-reward coefficient\.MethodTuned quantitiesConfigs per scaleSelected setting \(3B / 7B\)Search\-G1λg,η,K,M\\lambda\_\{g\},\\eta,K,M12\(0\.6,0\.02,25,300\)\(0\.6,0\.02,25,300\)/ sameSearch\-R1direct bonus; KL coefficient9\(0\.5,10−3\)\(0\.5,10^\{\-3\}\)/ sameGiGPOstate\-credit scale; anchor\-group cap9\(1\.0,5\)\(1\.0,5\)/ sameAgentPRMprocess\-reward scale; KL coefficient9\(1\.0,10−3\)\(1\.0,10^\{\-3\}\)/ sameGPT\-5 Judgesupport\-reward scale; KL coefficient9\(0\.5,10−3\)\(0\.5,10^\{\-3\}\)/ sameARPOentropy scale; allocation temperature9\(1\.0,1\.0\)\(1\.0,1\.0\)/ sameIGPOinformation\-gain scale; normalization9\(1\.0,group\)\(1\.0,\\text\{group\}\)/ sameSearch\-E1distillation mixture; temperature9\(0\.25,1\.0\)\(0\.25,1\.0\)/ sameSAASboundary threshold; allocation scale9\(0\.5,1\.0\)\(0\.5,1\.0\)/ sameβ\\beta\-GRPOconfidence threshold; reward scale9\(0\.5,1\.0\)\(0\.5,1\.0\)/ sameKbPOcertainty weight; quality weight9\(0\.5,0\.5\)\(0\.5,0\.5\)/ same

##### Reward coefficient sensitivity\.

We study sensitivity to the evidence\-reliance and search\-cost coefficients on 2WikiMultiHopQA\.[Table˜A13](https://arxiv.org/html/2608.07531#S7.T13)variesλg\\lambda\_\{g\}andη\\etaone at a time while keeping the other training settings fixed\. The default\(0\.6,0\.02\)\(0\.6,0\.02\)setting gives the highest displayed TC and lowest Search/Q at both scales, whereasλg=0\.8\\lambda\_\{g\}=0\.8gives higher EM but worse TC and Search/Q; increasingη\\etato0\.050\.05does not further reduce search\-marker use\. We do not sweepαcb\\alpha\_\{\\mathrm\{cb\}\}and fix it at0\.50\.5, matching the maximum correct\-direct bonus of the Search\-R1 reference; consequently, robustness toαcb\\alpha\_\{\\mathrm\{cb\}\}is not established by this table\. Equation[Equation˜7](https://arxiv.org/html/2608.07531#S3.E7)characterizes the local pre\-normalization incentive; the table reports downstream outcomes after retraining under each coefficient setting\.

Table A13:Reward\-coefficient sensitivity; entries are four\-run means\.λg\\lambda\_\{g\}η\\etaEM↑\\uparrowTCSearch/Q↓\\downarrowQwen2\.5\-3B\-Instruct0\.40\.020\.3470\.8542\.8420\.60\.020\.3680\.9722\.8120\.80\.020\.3810\.9113\.3830\.60\.000\.3380\.9203\.5760\.60\.050\.3340\.8913\.568Qwen2\.5\-7B\-Instruct0\.40\.020\.4570\.8782\.5210\.60\.020\.4750\.9782\.4950\.80\.020\.4860\.9262\.9800\.60\.000\.4490\.9343\.1440\.60\.050\.4460\.9093\.138
##### Estimator update sensitivity\.

We study how often the Search\-G1 estimators should be refit and how many refitting questions are needed at each update\. As the policy changes, hidden\-state distributions can drift away from the initial estimator data\.[Table˜A14](https://arxiv.org/html/2608.07531#S7.T14)evaluates the downstream effect of different co\-evolution schedules at both model scales on 2WikiMultiHopQA, whereKKis the refitting interval andMMis the number of refitting questions\. The defaultK=25,M=300K=25,M=300setting gives the highest displayed mean EM and TC;K=50K=50reduces Search/Q but slightly lowers both outcomes, whileK=10K=10orM=100M=100degrades the joint result\. Higher EM and lower Search/Q are preferred, with TC reporting the operational grounding response\.

Table A14:Estimator\-update sensitivity; entries are four\-run means\.Update ScheduleKKMMEM↑\\uparrowTCSearch/Q↓\\downarrowQwen2\.5\-3B\-InstructFrozen estimator∞\\infty00\.3570\.9063\.225Nominal 50\-update cadence503000\.3600\.9372\.656Nominal 25\-update cadence253000\.3680\.9722\.812Nominal 10\-update cadence103000\.3330\.8953\.589Nominal 25\-update cadence251000\.3390\.9123\.839Qwen2\.5\-7B\-InstructFrozen estimator∞\\infty00\.4660\.9222\.846Nominal 50\-update cadence503000\.4680\.9482\.362Nominal 25\-update cadence253000\.4750\.9782\.495Nominal 10\-update cadence103000\.4450\.9133\.155Nominal 25\-update cadence251000\.4500\.9273\.368

##### Cost\-controlled comparison\.

We score all methods on one fixed 1,024\-question 2WikiMultiHopQA subset under a unified accounting: identical question IDs, invalid trajectories charged EM=0\{=\}0, and Search/Q and per\-marker EM computed over all questions, while TC is conditioned on correct*searched*trajectories with effective sample sizes recorded in the manifest \([Table˜A15](https://arxiv.org/html/2608.07531#S7.T15)\)\. This searched\-only conditioning is deliberately stricter than the union\-of\-correct convention used for the full\-population grounding metrics in[Tables˜1](https://arxiv.org/html/2608.07531#S4.T1)and[3](https://arxiv.org/html/2608.07531#S4.T3): it removes direct\-answer trajectories from the TC denominator, so the TC values here are systematically lower and are*not*directly comparable to those tables\. Search/Q is unaffected by this conditioning—it is counted over all questions—but is likewise not directly comparable, as it is measured on this fixed 1,024\-question subset under the unified accounting \(invalid trajectories charged EM=0\{=\}0\) rather than on the full split\. We report this more conservative, unbiased conditioning to isolate grounding on trajectories that actually searched; even under it, Search\-G1 attains the highest all\-question EM, the highest conditioned TC, and the highest EM per search marker\. We do not claim a matched token budget on this 2Wiki subset: the per\-question token audit is measured only on NQ \([Tables˜A4](https://arxiv.org/html/2608.07531#S7.T4)and[A5](https://arxiv.org/html/2608.07531#S7.T5)\)\.

Table A15:Cost\-controlled comparison on one fixed 1,024\-question 2WikiMultiHopQA subset \(Qwen2\.5\-3B\) under a unified protocol \(identical question IDs; invalid trajectories charged EM=0\{=\}0; Search/Q over all questions\)\. TC here is conditioned on correct*searched*trajectories only—a stricter denominator than the union\-of\-correct TC in[Tables˜1](https://arxiv.org/html/2608.07531#S4.T1)and[3](https://arxiv.org/html/2608.07531#S4.T3)—so its absolute level is lower and not directly comparable across tables\. EM per marker is a secondary descriptive quantity\.MethodEM↑\\uparrowTC↑\\uparrowSearch/Q↓\\downarrowEM per marker↑\\uparrowPrompted Search0\.2310\.6081\.6020\.144Search\-R10\.2850\.5341\.9420\.147Search\-G10\.3550\.6551\.5800\.225

#### 7\.3\.3Case study

We provide selected qualitative question\-level summaries from Qwen2\.5\-3B\-Instruct on 2WikiMultiHopQA\. Each case summarizes representative outputs for a question on which both Prompted Search and the Search\-R1 baseline fail while Search\-G1 answers correctly\. The cases illustrate method\-level differences in task accuracy, evidence alignment, and search behavior; because of this selection rule, they are not population\-level comparative evidence\.

![Refer to caption](https://arxiv.org/html/2608.07531v1/case_study_2wiki.png)Figure A2:Selected qualitative cases on 2WikiMultiHopQA using Qwen2\.5\-3B\-Instruct\.These question\-level summaries show representative method outputs on cases where Prompted Search and the Search\-R1 baseline fail; they are not intended as population\-level evidence\.
#### 7\.3\.4Additional validity and robustness checks

##### Evidence\-reliance label audit\.

The evidence\-reliance label in[Equation˜1](https://arxiv.org/html/2608.07531#S3.E1)compares a sampled trajectory answer with a deterministic evidence\-ablated decode, so it can mix evidence sensitivity with decoding mismatch\. We therefore re\-derive the label under \(i\)*paired\-greedy*decoding and \(ii\)*common\-random\-number \(CRN\) paired sampling*, which couples the sampling variates to reduce independent decoding noise\. On Qwen2\.5\-3B\-Instruct/NQ at checkpoint 50, the original labels agree with these alternatives on 93%/96% of trajectories \(κ=0\.85/0\.90\\kappa=0\.85/0\.90\)\. The positive\-label rate changes by−0\.02/−0\.01\-0\.02/\-0\.01, while 6%/4% of group\-relative advantage signs flip \([Table˜A16](https://arxiv.org/html/2608.07531#S7.T16)\)\. Thus decoding mismatch is not the dominant source of the label in this audit, although its optimization effect is nonzero\.

Table A16:Evidence\-reliance decoding audit on Qwen2\.5\-3B\-Instruct/NQ under paired\-greedy and common\-random\-number interventions\.ConstructionAgree\.↑\\uparrowκ↑\\kappa\\uparrowPos\.\-rateΔ\\DeltaAdv\.\-sign flip↓\\downarrowPaired\-greedy0\.930\.85−0\.02\-0\.020\.06CRN paired sampling0\.960\.90−0\.01\-0\.010\.04

##### Shortcut\-control audit\.

The representation readout could exploit search count, response length, answer\-string overlap, or split leakage instead of evidence reliance\.[Table˜A17](https://arxiv.org/html/2608.07531#S7.T17)reports the matched shortcut\-control audit: every scalar control is fit on the same group\-disjoint split as the representation readout, and the representation result is repeated after exact matching on search\-count and response\-length bins\.

Table A17:Shortcut\-control audit for Qwen2\.5\-3B\-Instruct\. AUC is measured on the same held\-out deletion\-sensitivity labels as the main estimator audit\.Predictor/controlNQ AUC2Wiki AUCSearch count only0\.5480\.562Response length only0\.5570\.574Retrieved\-token count only0\.5360\.551Answer–evidence lexical overlap0\.5890\.603Strongest output/logit baseline0\.6380\.693Label\-shuffled representation0\.5010\.498Full answer\-commit representation0\.9590\.871Matched on search count and length0\.9340\.842

##### Retrieval\-necessity label audit\.

The necessity targetziz\_\{i\}in[Equation˜2](https://arxiv.org/html/2608.07531#S3.E2)comes from a deterministic closed\-book decode scored by normalized answer inclusion, which differs from the task exact\-match verifier\. On the same full held\-out Base\-model decodes used in[Table˜A3](https://arxiv.org/html/2608.07531#S7.T3), exact\-match correctness is a subset of inclusion sufficiency\. The two labels therefore disagree on249/3,610=6\.90%249/3\{,\}610=6\.90\\%of NQ examples and450/12,576=3\.58%450/12\{,\}576=3\.58\\%of 2Wiki examples\.

##### Necessity\-score distribution across refits\.

We recomputeni=1−bin\_\{i\}=1\-b\_\{i\}by applying each refitted boundary readout to its matched hidden\-state dump on the same 300\-question 2Wiki calibration slice\. The distribution is strongly bimodal: the 10th percentile remains near zero, while the median is above0\.930\.93and the 90th percentile is near one at every audited checkpoint \([Table˜A18](https://arxiv.org/html/2608.07531#S7.T18)\)\. Its mean changes non\-monotonically from0\.6170\.617to0\.6220\.622to0\.6050\.605, so the audit does not support a monotonic boundary shift\. These values characterize refitted readouts on their calibration slice rather than the online rollout\-time score distribution\.

Table A18:Necessity\-score distribution across refits on a fixed 300\-question 2WikiMultiHopQA calibration slice, Qwen2\.5\-3B\-Instruct\. Hard S/N are intervention\-label counts; Pred\. N is the number of readout scores withni≥0\.5n\_\{i\}\\geq 0\.5\. They need not agree because the readout is imperfect\.CheckpointHard S/NMeanSDq10 / median / q90Pred\. N \(%\\%\)AUC25115/1850\.6170\.4540\.011 / 0\.969 / 1\.000185 \(61\.67%\)0\.86650118/1820\.6220\.4300\.026 / 0\.932 / 1\.000183 \(61\.00%\)0\.88875122/1780\.6050\.4480\.013 / 0\.948 / 1\.000179 \(59\.67%\)0\.881

##### Common\-question unconditional behavior decomposition\.

TC and D\-Inv condition on method\-specific correct trajectories, so an unconditional common\-question analysis is needed to expose selection effects\. Using shared item IDs and support labels for all baselines,[Table˜A19](https://arxiv.org/html/2608.07531#S7.T19)reports the five\-method analysis on one fixed NQ set, keeps partial support separate, and reports unconditional grounded accuracyUGA=P​\(correct direct\)\+P​\(correct search, full\)\+0\.5​P​\(correct search, partial\)\\mathrm\{UGA\}=P\(\\text\{correct direct\}\)\+P\(\\text\{correct search, full\}\)\+0\.5P\(\\text\{correct search, partial\}\)\.

Table A19:Common\-question unconditional decomposition on a sharedN=600N=600NQ sample\. The five mutually exclusive buckets sum to one; UGA is reported with a question\-bootstrap 95% CI\.MethodCorr\.\-directSearch\-fullSearch\-partialSearch\-unsupp\.↓\\downarrowIncorrectUGA \[95% CI\]↑\\uparrowPrompted Search0\.100\.090\.050\.050\.710\.215​\[0\.184,0\.248\]0\.215\\,\[0\.184,0\.248\]Search\-R10\.090\.150\.070\.060\.630\.275​\[0\.240,0\.312\]0\.275\\,\[0\.240,0\.312\]GPT\-5 Judge0\.110\.210\.070\.030\.580\.355​\[0\.317,0\.394\]0\.355\\,\[0\.317,0\.394\]Search\-E10\.110\.190\.070\.040\.590\.335​\[0\.298,0\.373\]0\.335\\,\[0\.298,0\.373\]Search\-G10\.120\.200\.080\.030\.570\.360​\[0\.322,0\.399\]0\.360\\,\[0\.322,0\.399\]

On this shared question set Search\-G1 attains the highest unconditional grounded accuracy and, together with GPT\-5 Judge, the lowest unsupported rate; its unconditional grounded utility is thus competitive with the strongest external\-judge baseline, while its*conditional*support quality on matched questions \([Table˜A11](https://arxiv.org/html/2608.07531#S7.T11)\) is higher\. Search\-G1 therefore strengthens conditional grounding without sacrificing unconditional grounded utility, and it does so with an intrinsic reward rather than the external judge calls used by GPT\-5 Judge\.

##### Readout calibration audit\.

Because the readout scoresd,bd,benter the reward throughg=d​\(1−b\)g=d\(1\-b\), ranking quality \(AUC\) is not sufficient\. We therefore evaluate probabilistic calibration on a natural\-class\-balance held\-out set that is separate from layer/architecture selection\. The evidence/closed\-book readouts obtain AUC 0\.87/0\.80, Brier score 0\.14/0\.18, and ECE 0\.05/0\.07\. We define scale drift as the mean absolute change in the held\-out mean score between adjacent refits; the measured values are 0\.03/0\.04 \([Table˜A20](https://arxiv.org/html/2608.07531#S7.T20)\)\. These results provide no evidence of large score\-scale drift in this audited setting, but we do not interpretg=d​\(1−b\)g=d\(1\-b\)as a joint probability\.

Table A20:Readout\-calibration audit for the evidence\-dependence and closed\-book\-sufficiency readouts\.ReadoutAUC↑\\uparrowBrier↓\\downarrowECE↓\\downarrowScale drift across refits↓\\downarrowDϕD\_\{\\phi\}\(evidence\)0\.870\.140\.050\.03BψB\_\{\\psi\}\(closed\-book\)0\.800\.180\.070\.04

##### Co\-evolution stability diagnostics\.

We characterize co\-evolution stability on fixed anchor trajectories\. Reward shift is the mean absolute change in normalized shaping reward at a refresh, and performance jump is the absolute held\-out EM change across the surrounding evaluation windows\. AcrossK∈\{10,25,50\}K\\in\\\{10,25,50\\\}, both are larger atK=10K=10\(0\.12/0\.03\) than atK=25K=25or 50 \(0\.05–0\.06/0\.01\) \([Table˜A21](https://arxiv.org/html/2608.07531#S7.T21)\)\.

Table A21:Co\-evolution stability diagnostics across refit intervalsKK\.Refit intervalKKReward shift at refresh↓\\downarrowPerf\. jump↓\\downarrow10100\.120\.0325250\.060\.0150500\.050\.01

##### Credit\-usefulness control\.

Tie\-breaking capacity alone does not establish that the resulting credit is informative\. The score\-permutation control preserves each group’s reward multiset while disrupting trajectory–score alignment, and the cost\-only control removes reliance shaping\.[Table˜A22](https://arxiv.org/html/2608.07531#S7.T22)reports the sample sizes and uncertainty for these controls, including the cost\-only correlation and item\-level support annotations\.

Table A22:Credit\-usefulness controls on Qwen2\.5\-3B\-Instruct/2WikiMultiHopQA\. EM is mean±\\pmSD over four runs; support and correlations include 95% CIs\.Reward variantEval\.NNSupportNNEM↑\\uparrowHuman support↑\\uparrowAdv\.–support corr\.↑\\uparrowScore\-permutation control6001800\.350±0\.0120\.350\\pm 0\.0120\.630​\[0\.580,0\.680\]0\.630\\,\[0\.580,0\.680\]0\.02​\[−0\.13,0\.17\]0\.02\\,\[\-0\.13,0\.17\]Cost\-only6001800\.350±0\.0130\.350\\pm 0\.0130\.660​\[0\.610,0\.710\]0\.660\\,\[0\.610,0\.710\]0\.08​\[−0\.07,0\.23\]0\.08\\,\[\-0\.07,0\.23\]Search\-G16001800\.370±0\.0110\.370\\pm 0\.0110\.730​\[0\.680,0\.780\]0\.730\\,\[0\.680,0\.780\]0\.41​\[0\.28,0\.52\]0\.41\\,\[0\.28,0\.52\]

##### Cross\-dataset transfer baseline\.

To interpret the transfer results in[Table˜A9](https://arxiv.org/html/2608.07531#S7.T9), we compare Search\-G1 against the Search\-R1 baseline trained on the same NQ source and evaluated cross\-domain\. The point difference is\+0\.101\+0\.101in\-domain and only\+0\.008/\+0\.006/\+0\.001\+0\.008/\+0\.006/\+0\.001on HotpotQA/2Wiki/MuSiQue\. The paired\-bootstrap intervals in[Table˜A23](https://arxiv.org/html/2608.07531#S7.T23)are computed from the row\-level paired outputs; they show a significant in\-domain gain and cross\-domain performance that is competitive with outcome\-plus\-directness RL\.

Table A23:Cross\-dataset transfer against a source\-matched RL baseline \(Search\-R1 trained on NQ\)\. Method means, differences, and paired\-bootstrap 95% CIs are empirical\.†\\daggermarks an interval excluding zero\.Method \(train NQ\)NQHotpotQA2WikiMuSiQueSearch\-R10\.3520\.2400\.2500\.090Search\-G10\.4530\.2480\.2560\.091Δ\\Delta\[95% CI\]\+0\.101​\[\+0\.072,\+0\.130\]†\+0\.101\\,\[\+0\.072,\+0\.130\]^\{\\dagger\}\+0\.008​\[−0\.012,\+0\.028\]\+0\.008\\,\[\-0\.012,\+0\.028\]\+0\.006​\[−0\.015,\+0\.027\]\+0\.006\\,\[\-0\.015,\+0\.027\]\+0\.001​\[−0\.012,\+0\.014\]\+0\.001\\,\[\-0\.012,\+0\.014\]