SOAR：面向搜索智能体的来源感知强化学习与精准信用分配

# SOAR: Source-Aware Reinforcement Learning with Precise Credit Assignment for Search Agents
Source: [https://arxiv.org/html/2610.13157](https://arxiv.org/html/2610.13157)
###### Abstract

Reinforcement learning \(RL\) for search agents often suffers from evidence fabrication, a failure mode where agents bypass external information \(ℰe​x​t\\mathcal\{E\}\_\{ext\}\) and fabricates an incorrect answer\. We argue that this fabrication arises from an over\-reliance on unreliable internal knowledge \(ℰi​n​t\\mathcal\{E\}\_\{int\}\) as shortcuts due to the inherent opacity of reasoning trajectories\. To address this, we proposeSOAR\(SOurce\-AwareReinforcement learning\), a paradigm that transforms monolithic reasoning into source\-explicit trajectories\.SOAR\(i\) disentanglesℰe​x​t\\mathcal\{E\}\_\{ext\}andℰi​n​t\\mathcal\{E\}\_\{int\}into parallel, observable streams via a Source\-Aware Workflow, and \(ii\) incorporates Source\-Consistent Reward to guide the policy evolution\. By making information\-source attribution explicit,SOARenables precise credit assignment between different knowledge streams\. Evaluations show thatSOARoutperforms baselines on 3B and 7B backbones by7\.3%and5\.4%, while reducing evidence fabrication by74\.8%\. These confirm that SOAR significantly improves performance on knowledge\-intensive benchmarks while effectively mitigating evidence fabrication, fostering more faithful and grounded reasoning with enhanced robustness\.

11footnotetext:Equal contribution\.22footnotetext:Corresponding author\.33footnotetext:Project leader\.## 1Introduction

Figure 1:Comparison of reasoning trajectories and evidence fabrication rates\. \(a\) Previous methods often fall victim to evidence fabrication when initial retrieval results are incomplete, relying on hallucinated internal priors to fill gaps\. \(b\) Our method \(SOAR\) implements a source\-aware adjudication mechanism that monitors the alignment between internal knowledge \(ℰi​n​t\\mathcal\{E\}\_\{int\}\) and external evidence \(ℰe​x​t\\mathcal\{E\}\_\{ext\}\)\. By realizing the unreliability of internal information, the model initiates targeted retrieval to ensure a faithful reasoning trajectory\. \(c\) Quantitative analysis shows SOAR reduces the evidence fabrication rate to 4\.2%, outperforming existing baselines\.Large Language Models \(LLMs\) have demonstrated remarkable reasoning capabilities, yet they are inherently constrained by their static parametric knowledge[Hendrycks et al\. \(2020\)](https://arxiv.org/html/2610.13157#bib.bib39);[Wei et al\. \(2022\)](https://arxiv.org/html/2610.13157#bib.bib40);[Huang and Chang \(2023\)](https://arxiv.org/html/2610.13157#bib.bib41);[Schick et al\. \(2023\)](https://arxiv.org/html/2610.13157#bib.bib42);[Che et al\. \(2024\)](https://arxiv.org/html/2610.13157#bib.bib50)\. To mitigate hallucinations and handle knowledge\-intensive tasks, recent works have evolved from static RAG to autonomous search agents[Shi et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib13);[Jin et al\. \(2025b\)](https://arxiv.org/html/2610.13157#bib.bib4);[Yao et al\. \(2022\)](https://arxiv.org/html/2610.13157#bib.bib36);[Zheng et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib5)\. By leveraging Reinforcement Learning \(RL\), these agents learn to interact with external search engines, iteratively retrieving evidence to support multi\-step reasoning\. Representative frameworks show that such "think\-then\-search" paradigms can significantly enhance performance in open\-domain challenges\.

Despite their promise, there exists a critical failure mode in search agent RL training: Evidence Fabrication, where the agent’s reasoning evidence is not grounded in the retrieved information, as illustrated in Figure[1](https://arxiv.org/html/2610.13157#S1.F1)\(a\)\. While previous studies[Jia et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib28);[Xu et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib26);[Xinjie et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib27)have also identified this phenomenon, they often attribute it to limited model capacity or data biases, overlooking that the architectural limitations of search agentic RL frameworks can themselves induce evidence fabrication\. As quantified in Figure[1](https://arxiv.org/html/2610.13157#S1.F1)\(c\), we measure the Evidence Fabrication Rate across prior approaches,with Search\-R1[Jin et al\. \(2025b\)](https://arxiv.org/html/2610.13157#bib.bib4)reaching 16\.0% , followed by IKEA[Huang et al\. \(2025b\)](https://arxiv.org/html/2610.13157#bib.bib17)at 13\.4% and AutoRefine[Shi et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib13)at 11\.1%, highlighting this open issue\.

We argue that this phenomenon stems from an over\-reliance on internal knowledge \(ℰi​n​t\\mathcal\{E\}\_\{int\}\) as shortcuts\. This stems from the inherent opacity of monolithic reasoning trajectories developed in training, which fail to track whether the reasoning evidence comes from internal or external sources\. This creates a credit assignment flaw: the reward signal cannot distinguish whether a successful outcome is a result of grounded synthesis ofℰe​x​t\\mathcal\{E\}\_\{ext\}or an ungrounded heuristic fromℰi​n​t\\mathcal\{E\}\_\{int\}\. Consequently, the RL process fails to provide precise signals needed to suppress internal shortcuts, ultimately reinforcing unfaithful reasoning behaviors instead of authentic evidence utilization\.

To address this, we proposeSOAR\(SOurce\-AwareReinforcement Learning\)\.SOARintroduces two core innovations: \(i\) aSource\-Aware Workflowthat explicitly disentangles and organizes external information \(ℰe​x​t\\mathcal\{E\}\_\{ext\}\) and internal knowledge \(ℰi​n​t\\mathcal\{E\}\_\{int\}\) into observable streams, and \(ii\)Source\-Consistent Rewardthat enforces strict alignment between the model’s self\-declared evidence source and its reasoning process\.SOARmakes the reasoning evidence fully traceable, and ensures precise credit assignment across trajectories with different evidence sources during training\.

Our contributions are summarized as follows:

- •We proposeSOAR, an RL framework with aSource\-Aware Workflowthat disentanglesℰe​x​t\\mathcal\{E\}\_\{ext\}andℰi​n​t\\mathcal\{E\}\_\{int\}into observable streams to ensure precise credit assignment\.
- •We design aSource\-Consistent Rewardthat enforces alignment between the model’s self\-declared sources and its reasoning process, preventing reliance on internal shortcuts\.
- •Evaluations across seven benchmarks show that SOAR consistently outperforms competitive RL baselines, achieving average relative improvements of7\.3%\(3B\) and5\.4%\(7B\)\. Notably, SOAR leads to a74\.81%relative reduction in evidence fabrication, demonstrating that our framework successfully guides the model toward faithful evidence\-based reasoning instead of exploiting internal shortcuts\.

## 2Preliminary

### 2\.1Search Agents

Following prior work[Hu et al\. \(2018\)](https://arxiv.org/html/2610.13157#bib.bib48);[Jin et al\. \(2025b\)](https://arxiv.org/html/2610.13157#bib.bib4);[Yao et al\. \(2022\)](https://arxiv.org/html/2610.13157#bib.bib36);[Lewis et al\. \(2020\)](https://arxiv.org/html/2610.13157#bib.bib46), we study search agents where an LLM policyπθ\\pi\_\{\\theta\}interacts with external search tools to solve knowledge\-intensive tasks\. Given a queryxx, the agent produces a reasoning trajectory interleaved with search actions and retrieved observations, before generating the final answeryy\. Existing agentic RL approaches, typically based on GRPO\-style optimization[Shao et al\. \(2024\)](https://arxiv.org/html/2610.13157#bib.bib37), optimize the policy using answer\-level rewards:

maxπθ𝔼x,y∼πθ\[rϕ\(x,y\)\]−βDKL\(πθ∥πref\)\.\\max\_\{\\pi\_\{\\theta\}\}\\quad\\mathbb\{E\}\_\{x,y\\sim\\pi\_\{\\theta\}\}\\left\[r\_\{\\phi\}\(x,y\)\\right\]\-\\beta D\_\{\\mathrm\{KL\}\}\(\\pi\_\{\\theta\}\\,\\\|\\,\\pi\_\{\\text\{ref\}\}\)\.\(1\)
However, such sparse supervision creates a credit assignment flaw: with the evidence sources hidden within the monolithic trajectories, the reward cannot distinguish whether a answer is derived from grounded external evidence or from a spurious reliance on internal knowledge\.A comprehensive literature review on search agentic RL and faithful learning is detailed in Appendix[A](https://arxiv.org/html/2610.13157#A1)\.

### 2\.2Internal vs\. External Information

We categorize information into two types: \(i\)Internal Knowledge \(ℰi​n​t\\mathcal\{E\}\_\{int\}\), which comprises information implicitly stored within the model’s parameters and accessible without tool interaction; and \(ii\)External Information \(ℰe​x​t\\mathcal\{E\}\_\{ext\}\), representing task\-specific information retrieved via search tools\.

### 2\.3Challenge

We observe a systematic failure mode:Evidence Fabrication, as illustrated by the representative failure case in Figure[1](https://arxiv.org/html/2610.13157#S1.F1)\(a\)\.

Evidence Fabrication refers to a pathological pattern where the agent’s reasoning fundamentally departs from being grounded in the existing information context, particularly when the provided information lacks the necessary evidence to resolve the task\. This tendency is systematically reflected in the elevated fabrication rates observed in conventional RL\-based search agents \(Figure[1](https://arxiv.org/html/2610.13157#S1.F1)\(c\)\)\.

This behavior poses a fundamental threat to the operational reliability of search agents, undermining both the utility of retrieved evidence and the agent’s capacity for deep, persistent exploration when encountering complex tasks\.

Figure 2:The training framework of SOAR\.\(a\) Overall workflow: for a given questionqq, the Actor Model interacts with a search engine and its internal knowledge base to generate a group of diverse reasoning trajectories\{o1,o2,…,og\}\\\{o\_\{1\},o\_\{2\},\\dots,o\_\{g\}\\\}\. These trajectories are evaluated by a reference model and rule\-based rewards to compute the Group Advantage for reinforcement learning\. \(b\) Fine\-grained information selection: each trajectoryogo\_\{g\}represents a unique reasoning path derived from different selections and integration ofℰi​n​t\\mathcal\{E\}\_\{int\}andℰe​x​t\\mathcal\{E\}\_\{ext\}\. By contrasting these trajectories, the Group Advantage facilitates precise credit assignment to specific information selection behaviors, guiding the agent to learn optimal strategies for grounding its final answer\.

## 3Methodology

### 3\.1Motivation

While previous work might attribute evidence fabrication to limited model capacity, we argue that this pathological behavior is a consequence of over\-reliance on internal knowledge \(ℰi​n​t\\mathcal\{E\}\_\{int\}\) as shortcuts\. This stems from the inherent opacity of monolithic reasoning trajectories generated during training, which fail to track whether the reasoning evidence comes from internal or external sources\. This leads to abidirectional credit assignment flawin search agentic RL: \(i\) Positive Conflation, where the same reward is directly assigned to both shortcut\-driven and correct retrieval\-augmented paths, which dilutes the gradient signal for proper grounding; and \(ii\) Negative Ambiguity, where the lack of detailed attribution prevents the model from receiving precise feedback, as the objective cannot distinguish between internal knowledge gaps and incorrect external evidence synthesis\. Left unaddressed, this structural confusion encourages shortcut exploitation, causing the policy to fail to develop robust grounding mechanisms\(see Appendix[I](https://arxiv.org/html/2610.13157#A9)\)\.

To address this, we proposeSOAR\(SOurce\-AwareReinforcement Learning\), a paradigm that resolves the credit assignment flaw by enforcing rigorous reasoning provenance tracing throughout the policy optimization process\.

### 3\.2Source\-Aware Reinforcement Learning

#### 3\.2\.1Overview

We formulate the learning process as a source\-aware reinforcement learning problem \(see Figure[2](https://arxiv.org/html/2610.13157#S2.F2)\(a\)\)\. Building upon the GRPO framework, the policyπθ\\pi\_\{\\theta\}is trained to maximize the expected reward within a structured observation space:

maxπθ𝔼q∼𝒟,y∼πθ\(⋅∣q,ℰe​x​t,ℰi​n​t\)\[Rs​c\(q,y\)\]−βDKL\(πθ∥πref\),\\begin\{split\}\\max\_\{\\pi\_\{\\theta\}\}\\;\\mathbb\{E\}\_\{q\\sim\\mathcal\{D\},\\,y\\sim\\pi\_\{\\theta\}\(\\cdot\\mid q,\\mathcal\{E\}\_\{ext\},\\mathcal\{E\}\_\{int\}\)\}\\\!\\left\[R\_\{sc\}\(q,y\)\\right\]\\\\ \-\\beta\\,D\_\{\\mathrm\{KL\}\}\(\\pi\_\{\\theta\}\\\|\\pi\_\{\\mathrm\{ref\}\}\)\\end\{split\},\(2\)
whereℰe​x​t\\mathcal\{E\}\_\{ext\}andℰi​n​t\\mathcal\{E\}\_\{int\}denote the external evidence and the elicited internal knowledge streams produced by thesource\-aware workflow\. TheSource\-Consistent RewardRs​cR\_\{sc\}is designed to resolve the credit assignment ambiguity: it acts as a structural constraint that enforces strict alignment between each reasoning step and its self\-declared evidence provenance\. By conditioning the policy on these explicitly decoupled streams and optimizingRs​cR\_\{sc\}, SOAR reconfigures the optimization landscape to incentivize a accurate and faithful policy\.

#### 3\.2\.2Source\-Aware Workflow

To resolve the lack of evidence source traceability, we reconstruct the reasoning process intoSource\-Aware Workflow\. The core philosophy is to treat the model’s internal parameters as a dynamic, query\-contingent knowledge source that is accessed and recorded with the same structural rigor as an external corpus\. This systematic organization makes the use of internal information explicit and aligns it with that of external information, effectively transforming the unconstrained utilization of both internal and external information into transparent, attributable agentic actions\.

##### Context\-Isolated Internal Elicitation

Within this workflow, for each atomic queryqjq\_\{j\}generated, we initiate a dual\-stream information gathering process\. While a standard retrieverℛ\\mathcal\{R\}fetches external evidenceie​x​t\(j\)i\_\{ext\}^\{\(j\)\}from a corpus𝒟\\mathcal\{D\}, we simultaneously trigger a Pristine Self\-Elicitation to manifest the model’s internal state\.

To ensure thefunctional independenceof internal recall and prevent it from prematurely contaminating the final answer, this elicitation is conducted within aContext\-Isolated Environment\. We deliberately strip away the existing reasoning trajectoryτ\\tau, presenting only the specific queryqjq\_\{j\}to the agent:

ii​n​t\(j\)∼Pθ\(⋅∣Prompte​l​i​c,qj\),i\_\{int\}^\{\(j\)\}\\sim P\_\{\\theta\}\(\\cdot\\mid\\text\{Prompt\}\_\{elic\},q\_\{j\}\),\(3\)wherePθP\_\{\\theta\}denotes the probability distribution parameterized by the policy model, andPrompte​l​i​c\\text\{Prompt\}\_\{elic\}is a fixed, neutral template\. By restricting the conditioning set, we ensure thatii​n​t\(j\)i\_\{int\}^\{\(j\)\}is a high\-fidelity, explicit reflection of the model’s inherent parametric state, rather than a hallucination conditioned on the current reasoning context\.

##### Source\-Augmented Reasoning

Once streams are elicited, the workflow integrates them into a unified yet structured source\-aware observation, as showed in Figure[2](https://arxiv.org/html/2610.13157#S2.F2)\(b\)\. This symmetric presentation serves as a structural intervention that achieves two critical objectives for policy optimization:

Elimination of Attribution Ambiguity:By explicitly decoupling the conditioning space intoℰe​x​t\\mathcal\{E\}\_\{ext\}andℰi​n​t\\mathcal\{E\}\_\{int\}, we render the reasoning paths identifiable\. This allows the reinforcement learning process to distinguish whether a successful trajectory is grounded in internal knowledge or external retrieved evidence, resolving the inherent untraceability of evidence sources in monolithic trajectories\.

Precise Credit Assignment via Source Selection:Our framework facilitates an explicit adjudication between the elicited internal information and external evidence\. During the GRPO process, the policy gradient acts as a selection mechanism that evaluates thecomparative reliabilityof these sources\. If outputs generated using a specific information stream consistently receive higher advantagesA^\\hat\{A\}for a given query type, the policy learns to assign greater credit to that pathway\. This enables the policy to dynamically navigate the trade\-off between internal and external information, ultimately converging on a reasoning strategy that prioritizes the most robust source for each specific context\.

Through this explicit decoupling and comparative credit assignment,Source\-Aware Workflowensures that learning signals are properly attributed to distinct pathways, enabling the principled optimization of information\-acquisition strategies under reinforcement learning\. The detailed prompt templates are provided in Appendix[H](https://arxiv.org/html/2610.13157#A8)\.

#### 3\.2\.3Source\-Consistance Reward

To effectively guide policy learning, we adopt a multi\-dimensional reward functionRtotal=Rans⊕Rfmt⊕RmatR\_\{\\text\{total\}\}=R\_\{\\text\{ans\}\}\\oplus R\_\{\\text\{fmt\}\}\\oplus R\_\{\\text\{mat\}\}\. This formulation treatsRfmtR\_\{\\text\{fmt\}\}andRmatR\_\{\\text\{mat\}\}asstructural constraintsthat shape the optimization landscape, ensuring that terminal accuracy \(RansR\_\{\\text\{ans\}\}\) is optimized within the boundaries of verifiable reasoning\. Detailed reward definitions and coefficients are provided in Appendix[D](https://arxiv.org/html/2610.13157#A4)\.

##### \(i\) Answer Correctness Reward \(Ra​n​sR\_\{ans\}\)

Following[Jin et al\. \(2025a\)](https://arxiv.org/html/2610.13157#bib.bib38),Ra​n​sR\_\{ans\}provides the primary reinforcement signal for task completion\. We assign a binary reward based on the exact match between the parsed answer and the ground truthyg​ty\_\{gt\}:

Ra​n​s=\{1,ifextract​\(y\)=yg​t0,otherwise\.R\_\{ans\}=\\begin\{cases\}1,&\\text\{if \}\\text\{extract\}\(y\)=y\_\{gt\}\\\\ 0,&\\text\{otherwise\}\\end\{cases\}\.\(4\)

##### \(ii\) Format Compliance Reward \(Rf​m​tR\_\{fmt\}\)

To ensure the model adheres to the predefined agentic protocol \(e\.g\.,<search\>,<think\>tags\), we implement a rule\-basedRf​m​tR\_\{fmt\}\. This serves as a formal constraint and a necessary condition for valid trajectories during RL training:

Rf​m​t=α⋅𝕀⁡\(y​is structurally valid\)\.R\_\{fmt\}=\\alpha\\cdot\\mathbb\{I\}\(y\\text\{ is structurally valid\}\)\.\(5\)

##### \(iii\) Source\-Consistency Match Reward \(Rm​a​tR\_\{mat\}\)

Rm​a​tR\_\{mat\}enforces the faithfulness of the reasoning process by penalizing ambiguous outputs\. It ensures a strict alignment between the agent’s self\-declared evidence sourceCyC\_\{y\}of its evidence streamℰCy\\mathcal\{E\}\_\{C\_\{y\}\}:

Rm​a​t=γ⋅𝕀⁡\(Verify​\(y,ℰCy\)=1\),R\_\{mat\}=\\gamma\\cdot\\mathbb\{I\}\(\\text\{Verify\}\(y,\\mathcal\{E\}\_\{C\_\{y\}\}\)=1\),\(6\)whereVerify​\(⋅\)\\text\{Verify\}\(\\cdot\)evaluates whether the generated answer is consistent with the specified information stream\. By default, We adopt an exact\-match implementation for efficiency and compare it with an LLM\-based verifier in Appendix[B](https://arxiv.org/html/2610.13157#A2)\.

Table 1:Performance comparison ofSOARversus competitive baselines across diverse QA benchmarks\. The results represent Exact Match \(EM\) accuracy on Qwen2\.5\-3B and 7B backbones\. Bold denotes the best results\.Gray rows highlight our method and its relative improvement \(Δ%\\Delta\\%\) against the best\-performing baseline\.††nicematrix\-placeholder:NiceTabular\* \(nicematrix\)Table 2:System prompt for Evidence Fabrication Judge\.Task Instruction:Determine whether a model’s reasoning is grounded in the provided information\. Only judge whether the reasoning steps or the final answer actually depend on facts stated in the<information\>block\. Do NOT evaluate whether the final answer is correct\.Output Format:\{"reason": "\.\.\.","grounded": true/false\}

## 4EXPERIMENTS

### 4\.1Experimental Setting

DatasetsWe evaluate our method on diverse single\-hop and multi\-hop QA benchmarks, including single\-hop datasets: Natural Questions \(NQ\)[Kwiatkowski et al\. \(2019\)](https://arxiv.org/html/2610.13157#bib.bib29), TriviaQA[Joshi et al\. \(2017\)](https://arxiv.org/html/2610.13157#bib.bib31), PopQA[Mallen et al\. \(2023\)](https://arxiv.org/html/2610.13157#bib.bib30), and multi\-hop datasets: HotpotQA[Yang et al\. \(2018\)](https://arxiv.org/html/2610.13157#bib.bib32), 2WikiMultihopQA \(2Wiki\)[Ho et al\. \(2020\)](https://arxiv.org/html/2610.13157#bib.bib33), Musique[Trivedi et al\. \(2022\)](https://arxiv.org/html/2610.13157#bib.bib34), Bamboogle[Press et al\. \(2023\)](https://arxiv.org/html/2610.13157#bib.bib35)\. Exact match \(EM\) accuracy is used as the primary evaluation metric across all datasets\.We construct the training dataset by explicitly combining the training splits of NQ and HotpotQA\., allowing our agent RL to learn both single\-hop and multi\-hop reasoning patterns\.

Baseline MethodsTo assess the performance of our proposed SOAR framework, we select a comprehensive suite of baselines for comparative evaluation\. Specifically, we include Direct Inference and Naive RAG, which are derived from Search\-R1[Jin et al\. \(2025b\)](https://arxiv.org/html/2610.13157#bib.bib4)\(a work aligned with our experimental setup\), alongside Search\-R1[Jin et al\. \(2025b\)](https://arxiv.org/html/2610.13157#bib.bib4);[Jin et al\. \(2025a\)](https://arxiv.org/html/2610.13157#bib.bib38),AutoRefine[Shi et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib13), CoDA[Liu et al\. \(2025a\)](https://arxiv.org/html/2610.13157#bib.bib16), IKEA[Huang et al\. \(2025b\)](https://arxiv.org/html/2610.13157#bib.bib17), and AdaSearch[Lin et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib18)\. For fair comparison, we re\-implement Search\-R1, AutoRefine, CoDA, and IKEA via the publicly available code provided by their respective authors; the results of AdaSearch are adopted from its publication\.

Implementation DetailsWe use Qwen\-2\.5\-3B/7B\-Base[Qwen et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib43)as backbone agents, E5\-base\-v2[Wang et al\. \(2022\)](https://arxiv.org/html/2610.13157#bib.bib44)for retrieval, and the 2018 Wikipedia dump[Karpukhin et al\. \(2020\)](https://arxiv.org/html/2610.13157#bib.bib45)as the corpus\. The engine retrieves top\-3 relevant documents per query\. Models are trained via Group Relative Policy Optimization \(GRPO\)\.

Evaluation of Evidence Fabrication\.We employ GPT\-5\.6\-Sol as the judge to assess evidence fabrication\. We conduct the evaluation on the complete evaluation set and report results averaged over four independent runs\. Table[2](https://arxiv.org/html/2610.13157#S3.T2)presents the prompt template used for evidence fabrication evaluation\.

### 4\.2Main Results

We evaluateSOARacross diverse QA benchmarks, with additional Instruct\-tuned results in Appendix[E](https://arxiv.org/html/2610.13157#A5)\. Table[1](https://arxiv.org/html/2610.13157#S3.T1)leads to the following findings:

SOAR achieves state\-of\-the\-art performance across benchmarks and scales\.As shown in Table[1](https://arxiv.org/html/2610.13157#S3.T1),SOARconsistently outperforms competitive baselines on both 3B and 7B backbones\. On the 3B scale, it reaches an average of 0\.414, surpassingSearch\-R1by \+4\.2%\. The gains are most pronounced in multi\-hop reasoning: on Musique and 2Wiki,SOARsubstantially outperforms monolithic\-trajectory baselines, confirming that source transparency is critical for high\-entropy, multi\-step scenarios\. Furthermore, the training dynamics for the 3B model, as illustrated in Figure[3](https://arxiv.org/html/2610.13157#S4.F3)\(a\), show that the mean reward converges smoothly to a stable plateau\.

Consistent improvement with model scaling\.Performance scales reliably from 3B to 7B\. On the 7B backbone,SOARachieves an average of 0\.448, outperformingAutoRefine\(0\.425\) andIKEA\(0\.402\)\. Notably, on Bamboogle it improves from 0\.346 to 0\.453, demonstrating that resolving the credit assignment flaw provides a strong foundation for scaling search agents\.

Faithfulness drives performance\.As shown in Figure[1](https://arxiv.org/html/2610.13157#S1.F1)\(c\),SOARreduces the Evidence Fabrication Rate to 4\.05% \(a 74\.81% relative reduction\) on the 3B backbone\. In contrast to baselines that optimize for correctness without explicit grounding constraints,SOARensures that accuracy gains are driven by the effective selection and use of both internal and external knowledge, preventing the model from bypassing necessary retrieval with internal knowledge biases\.

Generalizability to harder QA\.SOARalso maintains clear advantages on the more challenging ASQA[Stelmakh et al\. \(2022\)](https://arxiv.org/html/2610.13157#bib.bib54)and WebWalkerQA[Wu et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib53)datasets; results details are provided in Appendix[F](https://arxiv.org/html/2610.13157#A6)\.

### 4\.3Model Behaviors

Figure 3:Analysis of model behaviors during training and inference\. \(a\) Mean reward convergence of SOAR and Search\-R1 over training steps\. \(b\) Search frequency dynamics during training: SOAR adaptively reduces evidence fabrication by shifting from internal shortcuts to proactive external search, as clearly reflected in the rising and stabilizing external retriever call rate\. \(c\) Search frequency across benchmarks: the search\-proactive policy learned during training generalizes consistently and robustly across diverse tasks\.Table 3:Ablation study on SOAR components\. The “w/o Workflow & Reward” row ablates both components\. Removing the Source\-Consistent Reward alone degrades performance; removing the Source\-Aware Workflow additionally \(i\.e\., removing both\) leads to a further degradation\. This confirms that both components are necessary\.We investigate the efficacy of SOAR in optimizing the agent’s information\-acquisition mechanism by resolving theCredit Assignment Flaw\.

Shift from Groundless Fabrication to Proactive Search\.As shown in Figure[3](https://arxiv.org/html/2610.13157#S4.F3)\(b\), the search frequency consistently increases during the early stages of training and eventually stabilizes at a robust higher plateau \(approximately 1\.6 calls per query\)\. By explicitly externalizingℰi​n​t\\mathcal\{E\}\_\{int\}through ourSource\-Aware Workflow, we expose the previously hidden shortcuts that lead to Evidence Fabrication\. This structural transparency allows theSource\-Consistance Rewardto precisely penalize instances where the agent improperly relies on its internal knowledge at the expense of grounded reasoning\. As the negative feedback loop suppresses these unreliable internal collapses, the agent is incentivized to seek more reliable evidence, resulting in a decisive strategic shift toward proactive search and a significantly more consistent external retrieval frequency as training progresses\.

Generalization of Search Behavior during Inference\.As illustrated in Figure[3](https://arxiv.org/html/2610.13157#S4.F3)\(c\), this learned preference for external verification successfully generalizes across various evaluation benchmarks\. Rather than being confined to the training distribution,SOARconsistently maintains an active search intensity across a wide spectrum of tasks, ranging from simple single\-hop queries to complex multi\-step reasoning problems\. This cross\-task consistency demonstrates that the search\-proactive policy, which acquired by resolving the credit assignment flaw during training, effectively transfers to inference\. This suggests that the model has internalized a source\-aware decision mechanism that prioritizes evidentiary grounding over parametric heuristics when encountering unfamiliar knowledge boundaries\. It ensures that the model’s reasoning remains anchored in verifiable evidence regardless of the specific evaluation context, confirming the generalizability of the information\-acquisition behavior\.

Table 4:Ablation of reward learning on evidence fabrication rate across different answer groups\.
### 4\.4Ablation Study

We conduct comprehensive ablation experiments using the Qwen2\.5\-3B backbone to decompose the respective contributions of theSource\-Aware WorkflowandSource\-Consistent Reward\. The results are summarized in Table[3](https://arxiv.org/html/2610.13157#S4.T3)and Table[4](https://arxiv.org/html/2610.13157#S4.T4)\.

Structural Transparency and Optimization Synergy\.As shown in Table[3](https://arxiv.org/html/2610.13157#S4.T3), the full configuration ofSOARconsistently achieves the highest performance across all multi\-hop benchmarks\. Removing both components \(w/o reward & workflow\) results in a significant performance decline from 0\.333 to 0\.294, confirming that structural transparency and precise optimization signals are both essential\. Notably, while the workflow alone provides the structural transparency required to resolve source ambiguity, it is the integration of our source\-consistent reward that pushes the model to its peak\. This synergy suggests that the workflow establishes a navigable reasoning space, while the reward mechanism provides the necessary directional pressure to steer the policy toward evidence\-grounded trajectories\. This is particularly evident in challenging tasks, where the reward mechanism effectively reshapes the optimization landscape to exploit rigorous, evidence\-based policy evolution\.

Reward as the Driver for Reasoning Faithfulness\.The impact of our reward shaping is even more pronounced in reducing evidence fabrication \(Table[4](https://arxiv.org/html/2610.13157#S4.T4)\)\. While the workflow provides the observable streams necessary for source separation, the reward mechanism is the primary driver that compels the model to actually utilize them\. By enforcing strict alignment between declarations and content, the reward mechanism further reduces the total fabrication rate from 6\.75% to 4\.05%\. Crucially, even inAnswer\-Correctinstances, the reward nearly halves the fabrication rate from 2\.75% to 1\.34%\. This indicates that the reward mechanism effectively penalizes spurious successes, where correct answers are bypassed as spurious hits if they rely on fabricated evidence\. By suppressing these ungrounded reasoning paths, the reward shaping compels the agent to prioritize faithfulness over mere accuracy, ensuring that correct answers are derived from verifiable evidence rather than unreliable internal hallucinations\.

Token Efficiency\.We compare the per\-turn generation cost betweenSOARand the strongest baseline\.SOARmaintains token efficiency comparable to baselines by parallelizing internal elicitation and external retrieval across GPUs\. Detailed results and implementation analysis are in Appendix[G\.1](https://arxiv.org/html/2610.13157#A7.SS1)\.

### 4\.5Systematic Robustness

To evaluate the stability of policy learning, we conduct robustness experiments where the training environment is intentionally corrupted with noise\.

Experimental Setup\.We inject noise into 5% of the training samples by targeting long\-tail queries where the model’s internal knowledge is sufficient\. For these samples, we replace the correct retrieved evidence with misleading information to create a reasoning conflict\. This setup evaluates whether the agent, when faced with localized contradictions representative of real\-world retrieval noise, will mistakenly retreat into unreliable internal priors or maintain its learned search\-proactive policy\.

Performance Resilience\.As shown in Table[5](https://arxiv.org/html/2610.13157#S4.T5),SOARexhibits superior robustness compared to baselines under adversarial pressure\. While Search\-R1 and IKEA suffer significant performance drops \(averaging \-0\.038\),SOARmaintains a stable performance with a decrease of only 0\.019\. This demonstrates thatSOARis less sensitive to noise, consistently identifying the optimal reasoning path even when specific training samples are corrupted\.

Faithfulness under Conflict\.Resilience ofSOARis further reflected in its reasoning integrity, as detailed in Table[6](https://arxiv.org/html/2610.13157#S4.T6)\. While noisy training drives baselines to distrust external sources, leading to a sharp increase in fabrication rate \(\+2\.15% forSearch\-R1\),SOAR’s fabrication rate remains constant \(\+0\.33%\)\. By resolving the credit assignment flaw,SOARensure the agent remains faithful to evidence despite environmental contradictions\.

Discussion\.These results confirm thatSOAReffectively captures the dominant signal of high\-quality retrieval\. Specifically, in our adversarial setup, since the corrupted queries could be answered by internal knowledge, baselines are easily misled into over\-relying on internal information\. This pathological optimization leads to increased fabrication rates\. Conversely,SOAR’s Source\-Aware architecture ensures the agent maintains its search\-proactive policy by filtering out noise and prioritizing the global grounding signal\.

Table 5:QA Stability in Corrupted Training\.Table 6:Evidence Fabrication in Corrupted Training\.

## 5Conclusion

This work critically examinesEvidence Fabricationin search agents, identifying thatmonolithic reasoningand the resultingcredit assignment flawdrive models to over\-rely on internal shortcuts over external evidence\. To address this, we proposeSOAR, which replaces opaque reasoning with aSource\-Aware Workflowto disentangleℰi​n​t\\mathcal\{E\}\_\{int\}andℰe​x​t\\mathcal\{E\}\_\{ext\}into observable streams\. By leveraging this transparency, ourSource\-Consistent Rewardresolves optimization ambiguity and enforces strict alignment between model claims and their underlying evidence\. Extensive evaluations across seven benchmarks confirm thatSOARsignificantly improves performance while fostering a grounded information\-acquisition policy, and does so at a controlled token cost relative to baselines\. Ultimately,SOARprovides a scalable approach for anchoring agents in verifiable evidence from information stream, ensuring reasoning integrity during complex policy evolution\.

## 6Limitations

We discuss the current limitations ofSOARand outline directions for future work\. Our primary training and evaluation rely on Wikipedia as a static, high\-quality knowledge source, which enables clean diagnosis of the Evidence Fabrication issue but does not fully capture the extreme volatility of open\-web environments\. To address this, we have provided a preliminary validation on ASQA and WebWalkerQA in Appendix[F](https://arxiv.org/html/2610.13157#A6), demonstrating thatSOARrobustly maintains its advantage under online search settings\. Nevertheless, testing the scaling limits of our source\-aware mechanism under more severe retrieval constraints—such as handling actively malicious misinformation or highly unstructured, conflicting web sources—remains an important direction for future study\.

## 7Ethics and Potential Risks

The primary goal of this work is to enhance the truthfulness of search\-based LLM agents\. By modeling information sources, our approach reduces hallucinations and encourages evidence\-grounded generation\. All experiments utilize public academic benchmarks with no known personally identifiable information \(PII\)\. Despite these benefits, potential risks remain due to the reliance on external retrieval\. Model outputs may inherit or amplify inaccuracies, biases, or incompleteness present in the underlying wiki corpus\. Future work will explore more robust verification mechanisms to ensure the reliability of retrieved evidence\.

## 8Acknowledgments

## References

- Cheet al\.\(2024\)T\. Che, X\. Mao, T\. Lan, and H\. HuangA hierarchical context augmentation method to improve retrieval\-augmented llms on scientific papers\.InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,pp\. 243–254\.Cited by:[§1](https://arxiv.org/html/2610.13157#S1.p1.1)\.
- Dinget al\.\(2025\)Y\. Ding, C\. Zhang, J\. Li, H\. Lin, X\. Liu, and M\. ZhangFAPO: flawed\-aware policy optimization for efficient and reliable reasoning\.arXiv preprint arXiv:2510\.22543\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1)\.
- Guiet al\.\(2026\)R\. Gui, Y\. Li, X\. Qu, Z\. Liu, Y\. Cheng, and Y\. ChengLearning to reason faithfully through step\-level faithfulness maximization\.arXiv preprint arXiv:2602\.03507\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1)\.
- Guoet al\.\(2025\)D\. Guo, D\. Yang, H\. Zhang, J\. Song, R\. Zhang, R\. Xu, Q\. Zhu, S\. Ma, P\. Wang, X\. Bi,et al\.Deepseek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1)\.
- Heet al\.\(2026\)B\. He, M\. Hu, Z\. Xu, H\. Wang, L\. Zong, Y\. Chen, C\. Ma, X\. Liu, P\. Zhou, and I\. KingSearch\-r2: enhancing search\-integrated reasoning via actor\-refiner collaboration\.arXiv preprint arXiv:2602\.03647\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1)\.
- Hendryckset al\.\(2020\)D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. SteinhardtMeasuring massive multitask language understanding\.arXiv preprint arXiv:2009\.03300\.Cited by:[§1](https://arxiv.org/html/2610.13157#S1.p1.1)\.
- Hoet al\.\(2020\)X\. Ho, A\. D\. Nguyen, S\. Sugawara, and A\. AizawaConstructing a multi\-hop qa dataset for comprehensive evaluation of reasoning steps\.arXiv preprint arXiv:2011\.01060\.Cited by:[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p1.1)\.
- Huet al\.\(2018\)Y\. Hu, Q\. Da, A\. Zeng, Y\. Yu, and Y\. XuReinforcement learning to rank in e\-commerce search engine: formalization, analysis, and application\.InProceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining,pp\. 368–377\.Cited by:[§2\.1](https://arxiv.org/html/2610.13157#S2.SS1.p1.1)\.
- Huang and Chang \(2023\)J\. Huang and K\. C\. ChangTowards reasoning in large language models: a survey\.InFindings of the association for computational linguistics: ACL 2023,pp\. 1049–1065\.Cited by:[§1](https://arxiv.org/html/2610.13157#S1.p1.1)\.
- Huanget al\.\(2025a\)L\. Huang, W\. Yu, W\. Ma, W\. Zhong, Z\. Feng, H\. Wang, Q\. Chen, W\. Peng, X\. Feng, B\. Qin,et al\.A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions\.ACM Transactions on Information Systems43\(2\),pp\. 1–55\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1)\.
- Huanget al\.\(2025b\)Z\. Huang, X\. Yuan, Y\. Ju, J\. Zhao, and K\. LiuReinforced internal\-external knowledge synergistic reasoning for efficient adaptive search agent\.arXiv preprint arXiv:2505\.07596\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1),[§1](https://arxiv.org/html/2610.13157#S1.p2.1),[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p2.1)\.
- Jaechet al\.\(2024\)A\. Jaech, A\. Kalai, A\. Lerer, A\. Richardson, A\. El\-Kishky, A\. Low, A\. Helyar, A\. Madry, A\. Beutel, A\. Carney,et al\.Openai o1 system card\.arXiv preprint arXiv:2412\.16720\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1)\.
- Jiaet al\.\(2025\)A\. S\. Jia, D\. Huang, N\. Vytla, N\. Choudhury, S\. Sen, J\. C\. Mitchell, and A\. DattaWhat is your agent’s gpa? a framework for evaluating agent goal\-plan\-action alignment\.arXiv preprint arXiv:2510\.08847\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1),[§1](https://arxiv.org/html/2610.13157#S1.p2.1)\.
- Jinet al\.\(2025a\)B\. Jin, J\. Yoon, P\. Kargupta, S\. O\. Arik, and J\. HanAn empirical study on reinforcement learning for reasoning\-search interleaved llm agents\.External Links:2505\.15117,[Link](https://arxiv.org/abs/2505.15117)Cited by:[§3\.2\.3](https://arxiv.org/html/2610.13157#S3.SS2.SSS3.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p2.1)\.
- Jinet al\.\(2025b\)B\. Jin, H\. Zeng, Z\. Yue, J\. Yoon, S\. Arik, D\. Wang, H\. Zamani, and J\. HanSearch\-r1: training llms to reason and leverage search engines with reinforcement learning\.arXiv preprint arXiv:2503\.09516\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1),[§1](https://arxiv.org/html/2610.13157#S1.p1.1),[§1](https://arxiv.org/html/2610.13157#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.13157#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p2.1)\.
- Joshiet al\.\(2017\)M\. Joshi, E\. Choi, D\. S\. Weld, and L\. ZettlemoyerTriviaqa: a large scale distantly supervised challenge dataset for reading comprehension\.arXiv preprint arXiv:1705\.03551\.Cited by:[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p1.1)\.
- Kaelblinget al\.\(1996\)L\. P\. Kaelbling, M\. L\. Littman, and A\. W\. MooreReinforcement learning: a survey\.Journal of artificial intelligence research4,pp\. 237–285\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1)\.
- Karpukhinet al\.\(2020\)V\. Karpukhin, B\. Oguz, S\. Min, P\. S\. Lewis, L\. Wu, S\. Edunov, D\. Chen, and W\. YihDense passage retrieval for open\-domain question answering\.\.InEMNLP \(1\),pp\. 6769–6781\.Cited by:[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p3.1)\.
- Kwiatkowskiet al\.\(2019\)T\. Kwiatkowski, J\. Palomaki, O\. Redfield, M\. Collins, A\. Parikh, C\. Alberti, D\. Epstein, I\. Polosukhin, J\. Devlin, K\. Lee,et al\.Natural questions: a benchmark for question answering research\.Transactions of the Association for Computational Linguistics7,pp\. 453–466\.Cited by:[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p1.1)\.
- Lewiset al\.\(2020\)P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.Advances in neural information processing systems33,pp\. 9459–9474\.Cited by:[§2\.1](https://arxiv.org/html/2610.13157#S2.SS1.p1.1)\.
- Li and Ng \(2025\)J\. Li and H\. T\. NgThe hallucination dilemma: factuality\-aware reinforcement learning for large reasoning models\.arXiv preprint arXiv:2505\.24630\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1)\.
- Liet al\.\(2025\)X\. Li, G\. Dong, J\. Jin, Y\. Zhang, Y\. Zhou, Y\. Zhu, P\. Zhang, and Z\. DouSearch\-o1: agentic search\-enhanced large reasoning models\.arXiv preprint arXiv:2501\.05366\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1)\.
- Lianget al\.\(2026a\)H\. Liang, Z\. Niu, J\. Wu, C\. Zhang, W\. Zhou, and J\. WangOThink\-srr1: search, refine and reasoning with reinforced learning for large language models\.arXiv preprint arXiv:2604\.19766\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1)\.
- Lianget al\.\(2026b\)Z\. Liang, Y\. Ma, B\. Chen, Z\. Qian, H\. Dai, L\. Mao, X\. Zhang, C\. Lei, and W\. OuIG\-search: step\-level information gain rewards for search\-augmented reasoning\.arXiv preprint arXiv:2604\.15148\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1)\.
- Linet al\.\(2024\)S\. Lin, L\. Gao, B\. Oguz, W\. Xiong, J\. Lin, W\. Yih, and X\. ChenFlame: factuality\-aware alignment for large language models\.Advances in Neural Information Processing Systems37,pp\. 115588–115614\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1)\.
- Linet al\.\(2025\)T\. Lin, W\. Chen, C\. Li, H\. Lee, Y\. Chen, and Y\. MengAdaSearch: balancing parametric knowledge and search in large language models via reinforcement learning\.arXiv preprint arXiv:2512\.16883\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1),[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p2.1)\.
- Liuet al\.\(2025a\)X\. Liu, J\. Feng, Z\. Zhuang, J\. Zhao, M\. Que, J\. Li, D\. Wang, H\. Tong, Y\. Chen, and P\. LiCoDA: a context\-decoupled hierarchical agent with reinforcement learning\.arXiv preprint arXiv:2512\.12716\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1),[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p2.1)\.
- Liuet al\.\(2026\)Y\. Liu, X\. Peng, Z\. Yan, Y\. Shen, W\. Xu, Y\. Huang, X\. Wang, J\. Cao, J\. Yin, and X\. ZhangDep\-search: learning dependency\-aware reasoning traces with persistent memory\.arXiv preprint arXiv:2601\.18771\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1)\.
- Liuet al\.\(2025b\)Z\. Liu, Y\. Zang, S\. Ding, Y\. Cao, X\. Dong, H\. Duan, D\. Lin, and J\. WangSPARK: synergistic policy and reward co\-evolving framework\.arXiv preprint arXiv:2509\.22624\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1)\.
- Mallenet al\.\(2023\)A\. Mallen, A\. Asai, V\. Zhong, R\. Das, D\. Khashabi, and H\. HajishirziWhen not to trust language models: investigating effectiveness of parametric and non\-parametric memories\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 9802–9822\.Cited by:[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p1.1)\.
- Presset al\.\(2023\)O\. Press, M\. Zhang, S\. Min, L\. Schmidt, N\. A\. Smith, and M\. LewisMeasuring and narrowing the compositionality gap in language models\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 5687–5711\.Cited by:[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p1.1)\.
- Qwenet al\.\(2025\)Qwen, :, A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. QiuQwen2\.5 technical report\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p3.1)\.
- Renet al\.\(2025\)B\. Ren, S\. Qiao, D\. Zheng, H\. Chen, and N\. ZhangKnowrl: exploring knowledgeable reinforcement learning for factuality\.arXiv preprint arXiv:2506\.19807\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1)\.
- Schicket al\.\(2023\)T\. Schick, J\. Dwivedi\-Yu, R\. Dessì, R\. Raileanu, M\. Lomeli, E\. Hambro, L\. Zettlemoyer, N\. Cancedda, and T\. ScialomToolformer: language models can teach themselves to use tools\.Advances in Neural Information Processing Systems36,pp\. 68539–68551\.Cited by:[§1](https://arxiv.org/html/2610.13157#S1.p1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.External Links:2402\.03300,[Link](https://arxiv.org/abs/2402.03300)Cited by:[§2\.1](https://arxiv.org/html/2610.13157#S2.SS1.p1.1)\.
- Shiet al\.\(2025\)Y\. Shi, S\. Li, C\. Wu, Z\. Liu, J\. Fang, H\. Cai, A\. Zhang, and X\. WangSearch and refine during think: autonomous retrieval\-augmented reasoning of llms\.arXiv preprint arXiv:2505\.11277\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1),[§1](https://arxiv.org/html/2610.13157#S1.p1.1),[§1](https://arxiv.org/html/2610.13157#S1.p2.1),[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p2.1)\.
- Snyderet al\.\(2024\)B\. Snyder, M\. Moisescu, and M\. B\. ZafarOn early detection of hallucinations in factual question answering\.InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,pp\. 2721–2732\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1)\.
- Stelmakhet al\.\(2022\)I\. Stelmakh, Y\. Luan, B\. Dhingra, and M\. ChangASQA: factoid questions meet long\-form answers\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,pp\. 8273–8288\.Cited by:[§4\.2](https://arxiv.org/html/2610.13157#S4.SS2.p5.1)\.
- Tamoet al\.\(2026\)J\. B\. Tamo, Y\. Lu, B\. L\. Marteau, M\. C\. Nnamdi, and M\. D\. WangEvidenceRL: reinforcing evidence consistency for trustworthy language models\.arXiv preprint arXiv:2603\.19532\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1)\.
- Tonmoyet al\.\(2024\)S\. Tonmoy, S\. Zaman, V\. Jain, A\. Rani, V\. Rawte, A\. Chadha, and A\. DasA comprehensive survey of hallucination mitigation techniques in large language models\.arXiv preprint arXiv:2401\.013136\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1)\.
- Trivediet al\.\(2022\)H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. SabharwalMuSiQue: multihop questions via single\-hop question composition\.Transactions of the Association for Computational Linguistics10,pp\. 539–554\.Cited by:[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p1.1)\.
- Wanget al\.\(2022\)L\. Wang, N\. Yang, X\. Huang, B\. Jiao, L\. Yang, D\. Jiang, R\. Majumder, and F\. WeiText embeddings by weakly\-supervised contrastive pre\-training\.arXiv preprint arXiv:2212\.03533\.Cited by:[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p3.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[§1](https://arxiv.org/html/2610.13157#S1.p1.1)\.
- Wenet al\.\(2026\)T\. Wen, G\. Dong, and Z\. DouSmartsearch: process reward\-guided query refinement for search agents\.arXiv preprint arXiv:2601\.04888\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1)\.
- Wuet al\.\(2025\)J\. Wu, W\. Yin, Y\. Jiang, Z\. Wang, Z\. Xi, R\. Fang, L\. Zhang, Y\. He, D\. Zhou, P\. Xie,et al\.Webwalker: benchmarking llms in web traversal\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 10290–10305\.Cited by:[§4\.2](https://arxiv.org/html/2610.13157#S4.SS2.p5.1)\.
- Xinjieet al\.\(2025\)Z\. Xinjie, F\. Gao, X\. Song, Y\. Chen, R\. Yang, Y\. Fu, Y\. Wang, Y\. Iwasawa, Y\. Matsuo, and I\. LiReagent: reversible multi\-agent reasoning for knowledge\-enhanced multi\-hop qa\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 4067–4089\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1),[§1](https://arxiv.org/html/2610.13157#S1.p2.1)\.
- Xuet al\.\(2026\)Y\. Xu, S\. Lu, J\. Cheng, M\. Wang, Q\. Xie, X\. Wang, R\. He, and J\. LiangHow to train your deep research agent? prompt, reward, and policy optimization in search\-r1\.arXiv preprint arXiv:2602\.19526\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1)\.
- Xuet al\.\(2024\)Y\. Xu, T\. Cai, J\. Jiang, and X\. SongFace4rag: factual consistency evaluation for retrieval augmented generation in chinese\.InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,pp\. 6083–6094\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1)\.
- Xuet al\.\(2025\)Z\. Xu, Z\. Wu, Y\. Zhou, A\. Feng, K\. Zhou, S\. Woo, K\. Ramnath, Y\. Tian, X\. Qi, W\. Qiu,et al\.Beyond correctness: rewarding faithful reasoning in retrieval\-augmented generation\.arXiv preprint arXiv:2510\.13272\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1),[§1](https://arxiv.org/html/2610.13157#S1.p2.1)\.
- Yanget al\.\(2018\)Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. Cohen, R\. Salakhutdinov, and C\. D\. ManningHotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 conference on empirical methods in natural language processing,pp\. 2369–2380\.Cited by:[§4\.1](https://arxiv.org/html/2610.13157#S4.SS1.p1.1)\.
- Yaoet al\.\(2022\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. CaoReact: synergizing reasoning and acting in language models\.InThe eleventh international conference on learning representations,Cited by:[§1](https://arxiv.org/html/2610.13157#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.13157#S2.SS1.p1.1)\.
- Zhanget al\.\(2026\)Z\. Zhang, A\. Lu, Y\. Zeng, Z\. Shan, J\. Guo, L\. Li, Y\. Li, and K\. RenGrad2Reward: from sparse judgment to dense rewards for improving open\-ended llm reasoning\.arXiv preprint arXiv:2602\.01791\.Cited by:[§A\.2](https://arxiv.org/html/2610.13157#A1.SS2.p1.1)\.
- Zhenget al\.\(2018\)G\. Zheng, F\. Zhang, Z\. Zheng, Y\. Xiang, N\. J\. Yuan, X\. Xie, and Z\. LiDRN: a deep reinforcement learning framework for news recommendation\.InProceedings of the 2018 world wide web conference,pp\. 167–176\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1)\.
- Zhenget al\.\(2025\)Y\. Zheng, D\. Fu, X\. Hu, X\. Cai, L\. Ye, P\. Lu, and P\. LiuDeepresearcher: scaling deep research via reinforcement learning in real\-world environments\.arXiv preprint arXiv:2504\.03160\.Cited by:[§A\.1](https://arxiv.org/html/2610.13157#A1.SS1.p1.1),[§1](https://arxiv.org/html/2610.13157#S1.p1.1)\.

## Appendix ARelated Work

### A\.1RL for Search Agents

Reinforcement learning \(RL\) has been widely adopted to enhance the reasoning and decision\-making capabilities of large language models \(LLMs\)[Kaelbling et al\. \(1996\)](https://arxiv.org/html/2610.13157#bib.bib1);[Jaech et al\. \(2024\)](https://arxiv.org/html/2610.13157#bib.bib2);[Guo et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib3);[Zheng et al\. \(2018\)](https://arxiv.org/html/2610.13157#bib.bib49)\. By treating search calls as actions and optimizing outcome\-based rewards, representative approaches such as Search\-R1[Jin et al\. \(2025b\)](https://arxiv.org/html/2610.13157#bib.bib4), Search\-o1[Li et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib6), DeepResearcher[Zheng et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib5), Search\-R1\+\+[Xu et al\. \(2026\)](https://arxiv.org/html/2610.13157#bib.bib7), Search\-R2[He et al\. \(2026\)](https://arxiv.org/html/2610.13157#bib.bib8)and IG\-Search[Liang et al\. \(2026b\)](https://arxiv.org/html/2610.13157#bib.bib14)demonstrate that RL can effectively learn joint search\-and\-reasoning behaviors without explicit supervision over intermediate steps\. Subsequent studies focus on improving reasoning ability through structured deployments or reward shaping that encourage the use of adaptive and economical tools[Shi et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib13);[Liu et al\. \(2025a\)](https://arxiv.org/html/2610.13157#bib.bib16);[Liang et al\. \(2026a\)](https://arxiv.org/html/2610.13157#bib.bib12)\. Beyond general agents, several recent works like IKEA[Huang et al\. \(2025b\)](https://arxiv.org/html/2610.13157#bib.bib17), AdaSearch[Lin et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib18)and Dep\-Search[Liu et al\. \(2026\)](https://arxiv.org/html/2610.13157#bib.bib9), explicitly study how to balance internal knowledge and external retrieval to improve efficiency by delineating knowledge boundaries\. However, these methods primarily treat the use of internal versus external knowledge as a static decision or focus on search frequency\. This leaves a blind spot where trajectories dominated by internal parametric knowledge may receive high rewards even when external evidence is ignored, encouraging shortcut behaviors and evidence fabrication\.

In contrast, our work focuses on the dynamic selection and utilization of knowledge sources within the rollout process\. By making knowledge\-source attribution explicit,SOARallows reinforcement learning to optimize over distinct knowledge\-dependent decision paths rather than search efficiency alone, resolving the credit assignment flaw between internal and external sources\.

### A\.2Faithful Learning in RL

In LLMs RL\-based post\-training, shortcut behaviors are often manifested as factually inconsistent hallucinations or shallow generation patterns that correlate superficially with reward signals, rather than reflecting faithful reasoning[Snyder et al\. \(2024\)](https://arxiv.org/html/2610.13157#bib.bib51);[Tonmoy et al\. \(2024\)](https://arxiv.org/html/2610.13157#bib.bib24);[Huang et al\. \(2025a\)](https://arxiv.org/html/2610.13157#bib.bib25);[Lin et al\. \(2024\)](https://arxiv.org/html/2610.13157#bib.bib23);[Gui et al\. \(2026\)](https://arxiv.org/html/2610.13157#bib.bib10)\. To alleviate these issues, prior work typically relies on output\-level supervision and verification, such as preference modeling, reward verification, and self\-consistency checks, which are usually implemented through external models or heuristic judges[Ren et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib20);[Li and Ng \(2025\)](https://arxiv.org/html/2610.13157#bib.bib19);[Ding et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib21);[Liu et al\. \(2025b\)](https://arxiv.org/html/2610.13157#bib.bib22);[Xu et al\. \(2024\)](https://arxiv.org/html/2610.13157#bib.bib52);[Tamo et al\. \(2026\)](https://arxiv.org/html/2610.13157#bib.bib11);[Zhang et al\. \(2026\)](https://arxiv.org/html/2610.13157#bib.bib15)\. In search agent RL scenarios, existing approaches to faithful learning in this context typically rely on external judges or verifier models to assess trajectory quality and evidence alignment[Jia et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib28);[Xu et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib26);[Xinjie et al\. \(2025\)](https://arxiv.org/html/2610.13157#bib.bib27);[Wen et al\. \(2026\)](https://arxiv.org/html/2610.13157#bib.bib47)\. While effective, such designs introduce additional training overhead and depend on the robustness of auxiliary models\.

Our proposedSOARis motivated by this critical and nuanced distinction\. We address the issue by making knowledge\-source attribution explicit within the rollout process, thereby transforming the monolithic reasoning process into a verifiable and structurally source\-transparent trajectory without relying on external judges\.

Table 7:Ablation of the Source\-Consistency Match Reward: exact\-match \(EM\) check vs\. LLM judge \(Claude Opus 4\.8\)\. Results are reported as mean±\\pmstd over 4 runs\.

## Appendix BImplementation of Verification

We implementVerify​\(y,ℰCy\)\\text\{Verify\}\(y,\\mathcal\{E\}\_\{C\_\{y\}\}\)using a rule\-based exact\-match \(EM\) verifier for computational efficiency during RL training\. To validate this design choice, we replace the EM verifier with an LLM\-based verifier \(Claude Opus 4\.8\) within the Source\-Consistency Match Reward and repeat training under identical settings\.

As shown in Table[7](https://arxiv.org/html/2610.13157#A1.T7), the LLM\-based verifier yields only marginal improvements over EM verification \(0\.4187 vs\. 0\.4160 average performance\), indicating that EM verification captures most source\-consistency signals while avoiding the substantially higher token cost and inference latency of LLM\-based verification\. We therefore adopt EM verification throughout all experiments\.

## Appendix CExperimental Details

In this appendix, we provide additional implementations, dataset configurations, and environment setups to further ensure the reproducibility of our work\. All revisions suggested through AI assistance were carefully reviewed and verified by the authors\.

### C\.1Dataset Configurations and Statistics

We evaluate our framework across multiple standard benchmarks\. The detailed statistics of the datasets, including their specific splits and sample counts for both the training and evaluation phases, are comprehensively summarized in Table[8](https://arxiv.org/html/2610.13157#A3.T8)\.

Table 8:Detailed sample statistics and configurations of the datasets used for training and evaluation phases\.PhaseDatasetNumber of ExamplesTrainHotpotQA90,447Natural Questions \(NQ\)79,168Total Train169,615TestPopQA14,2672WikiMultihopQA12,576TriviaQA11,313HotpotQA7,405Natural Questions \(NQ\)3,610MuSiQue2,417Bamboogle125Total Test51,713
### C\.2Hyperparameters and Infrastructure

Our training pipeline is implemented using theveRLframework\. We employ the Group Relative Policy Optimization \(GRPO\) algorithm to fine\-tune the Qwen2\.5 model family \(including Qwen2\.5\-3B, Qwen2\.5\-3B\-Instruct, and Qwen2\.5\-7B\) on a single node with 8×\\timesH200 NVIDIA GPUs\.

The training is performed for 3 epochs with a global batch size of 256 prompts per step\. The actor network is optimized using a learning rate of5×10−75\\times 10^\{\-7\}with a linear warmup ratio of 0\.285\. For the GRPO policy rollout, we sample 5 rollout outputs per prompt at a temperature of 1\.0 to compute group\-relative rewards\. The environment supports up to 5 search turns, utilizing an external retriever configured to fetch the top 3 relevant contexts\.

## Appendix DReward Configuration

Following Search\-R1\-v0\.3, the final reward is computed as

R=Rans\+α​Rfmt\+γ​Rmat,R=R\_\{\\text\{ans\}\}\+\\alpha R\_\{\\text\{fmt\}\}\+\\gamma R\_\{\\text\{mat\}\},\(7\)whereRansR\_\{\\text\{ans\}\}denotes the answer correctness reward,RfmtR\_\{\\text\{fmt\}\}denotes the format compliance reward, andRmatR\_\{\\text\{mat\}\}denotes the proposed Source\-Consistency Match Reward\. We setα=0\.2\\alpha=0\.2andγ=0\.1\\gamma=0\.1in all experiments\. The final reward is obtained by summing the three components and applying normalization\.

## Appendix EPerformance on Instruct\-tuned Backbone

To further verify the generalizability ofSOAR, we evaluate its performance using theQwen2\.5\-3B\-Instructmodel as the backbone\. As shown in Table[9](https://arxiv.org/html/2610.13157#A5.T9),SOARcontinues to demonstrate superior performance over competitive baselines\. Notably, it achieves leading results in complex multi\-hop tasks like2Wiki\(0\.411\) andMusique\(0\.144\), resulting in a leading average score of0\.397\.

Table 9:Performance comparison on theQwen2\.5\-3B\-Instructbackbone\. Bold denotes the best results\.
## Appendix FPerformance on Challenge benchmark

To further evaluateSOARunder more complex conditions, we conduct experiments on ASQA and WebWalkerQA, two benchmarks that require synthesizing information from multiple documents and involve longer reasoning chains\.

Experimental Setup\.For ASQA, we use E5\-base\-v2 as the retriever, with the 2018 Wikipedia dump as the corpus, retrieving the top\-3 relevant documents per query\. For WebWalkerQA, we migrateSOARto an online search setting, where the agent queries a live search API and receives the top\-3 returned results per query, mirroring the retrieval budget of the offline setting\.

Results\.As shown in Table[10](https://arxiv.org/html/2610.13157#A6.T10),SOARconsistently outperforms competitive baseline on both benchmarks\. The performance advantage observed in the main experiments transfers robustly to these harder, long\-form QA settings, confirming that the workflow generalizes effectively to real\-world online search and complex reasoning scenarios\.

Table 10:Zero\-shot performance comparison on challenging long\-form QA benchmarks using the Qwen2\.5\-7B backbone\. Evaluation metrics include F1 score for ASQA and Accuracy judged by GPT\-4o\-mini for WebWalkerQA\. Bold text indicates the best performance\.
## Appendix GAdditional Analysis

### G\.1Token Cost Analysis

We compare per\-turn generation efficiency betweenSOARand the strongest baseline\. As shown in Table[11](https://arxiv.org/html/2610.13157#A7.T11), Our method produces an average of 431 output tokens per turn, compared to 408 tokens for the baseline Search\-R1\. The slight increase stems from the explicit internal knowledge elicitation step, and we mitigate this overhead through two design choices\. First, we strictly control the length and quantity of internal knowledge: all noisy context is removed and only the essential entities are retained, substantially reducing the tokens cost in this step\. Second, internal elicitation and external retrieval are executed in parallel; moreover, we leverage the veRL framework to generate internal knowledge concurrently across GPUs\. These optimizations ensure thatSOARachieves its performance gains with only a marginal increase in token cost and no additional peak VRAM usage relative to the baseline\.

Table 11:Performance and efficiency trade\-off on 3B backbone\.Accdenotes the average score across the 7 main benchmarks\.Token Costrefers to the average output tokens per reasoning turn, encompassing both information and thinking content\.
### G\.2Statistical Significance Analysis

To verify that the improvements of SOAR are statistically reliable rather than artifacts of sampling noise, we run inference for both Search\-R1 and SOAR under four random seeds with a sampling temperature of1\.01\.0, and report the mean and standard deviation in Table[12](https://arxiv.org/html/2610.13157#A7.T12)\. The run\-to\-run variance is consistently small for both methods \(average standard deviation≤0\.0021\\leq 0\.0021\)\. In contrast, SOAR improves over Search\-R1 by\+4\.74\+4\.74points on the average score \(0\.41870\.4187vs\.0\.37130\.3713\), which is more than an order of magnitude larger than the observed variance\. Moreover, SOAR outperforms Search\-R1 on every benchmark across all seeds\. These results confirm that the gains of SOAR are stable and consistent\.

Table 12:Performance of Search\-R1 and SOAR under four random seeds \(temperature=1\.0=1\.0\) on the Qwen2\.5\-3B backbone\. Results are reported as mean±\\pmstd over 4 runs\. Bold denotes the best results\.
### G\.3Human Verification of Fabrication Evaluation

We conduct a human verification to examine the reliability of the automatic evaluation\. Specifically, we randomly sample 200 judge decisions from the evaluation outputs and manually verify whether each decision correctly identifies the presence or absence of evidence fabrication according to the same criteria as the judge prompt\. The judge agrees with human annotation in 98\.5% of the cases \(197 out of 200\), indicating that the automatic fabrication evaluation is highly consistent with human judgment and that the reported fabrication rates are reliable\.

## Appendix HAgent Prompt Templates

### H\.1Agent RL Training Prompt

Table[13](https://arxiv.org/html/2610.13157#A8.T13)details the system prompt used for Agent RL training, which guides the agent’s interaction with the external environment during the reinforcement learning process\.

Table 13:System prompt used for Agent RL training\.You are an expert assistant capable of solving knowledge\-intensive tasks efficiently\. You will be given a question to answer as accurately as possible by carefully balancingINTERNALinformation andEXTERNALinformation\.If you find you lack some knowledge, you can call an information engine by<search\> query </search\>\. It will return the top internal and external information\.To arrive at the answer, you will proceed step\-by\-step in a structured cycle of<think\>,<search\>,<information\>, and<information\_source\>sequences\. At end, you should use<answer\>to provide answer\. You can only generate content within these special tags\.Thinking Phase \(<think\>\):For question, it may be decomposed into sub\-questions for you to think about\. Some sub\-questions may be answered by searching, while others may not\. You can also use the<think\>tag to express your uncertainty about the sub\-question, weigh the accuracy and usefulness of both types of information, and decide whether further search is needed\.Searching Phase \(<search\>\):Formulate a search query only if required to fill a knowledge gap or verify uncertainty\.Information Phase \(<information\>\):Use internal and external results as context for further steps\.Information Source \(<information\_source\>\):Before providing your final answer, indicate clearly whether you are using your own internal knowledge or external information\. Only outputeither “internal” or “external”inside this tag, nothing else\.Answering Phase \(<answer\>\):Provide a concise and accurate answer within tags once you have enough knowledge\. The answer should be short and precise, such as<answer\> Beijing </answer\>\.Here are some examples:Example 1: search is needed, search more than once\.Question: xxx<think\> xxx </think\> <search\> xxx </search\> <information\> internal: xxx external: xxx </information\> … \(search more than once\) <information\_source\> internal/external </information\_source\> <answer\> xx </answer\>Example 2: search is needed, only search once\.Question: xxx?<think\> xxx </think\> <search\> xxx </search\> <information\> internal: xxx external: xxx </information\> <information\_source\> internal/external </information\_source\> <answer\> xx </answer\>Each search should be focused on one sub\-question\. The answer within<answer\>tags should be short and precise, such as<answer\> yes </answer\>\.Now it is your turn to answer the question\.Question:\{question\}
### H\.2Internal Knowledge Extraction Prompt

Table[14](https://arxiv.org/html/2610.13157#A8.T14)presents the prompt template used to extractℰi​n​t\\mathcal\{E\}\_\{int\}\. This prompt enforces a strict format to ensure that the internal knowledge is captured as a set ofkkindependent, declarative facts, providing a structured basis for the subsequent source\-consistency evaluation\.

Table 14:System prompt used for Internal Knowledge Extraction\.You are a helpful assistant\.Your task is to list knowledge relevant to the question usingONLYyour internal knowledge\. DoNOTinvent, guess, speculate, or hallucinate any facts\. If your internal knowledge is limited, state only what is confidently known\.You must STRICTLY follow all rules below:1\) Your outputMUSTcontain exactly\{top\-kk\}items\.2\) Each itemMUSTbe a single declarative sentence\.3\) Each itemMUSTbe formatted as a numbered bullet\.4\) OutputONLYthe\{top\-kk\}numbered bullets and nothing else\.5\) DoNOTinclude explanations, reasoning, summaries, or meta commentary\.

## Appendix ICase study

### I\.1Spurious Correctness from evidence fabrication

To further analyze model behaviors beyond final\-answer accuracy, we examine cases where the baseline\(Search\-R1\) produces the correct answer despite failing to retrieve sufficient supporting evidence\. Table[15](https://arxiv.org/html/2610.13157#A11.T15)presents a representative example\.

We observe thatSearch\-r1occasionally reaches the correct prediction without retrieving key evidence required for solving the question\. Instead, the model fabricates evidence to fill missing information\. Although the final answer happens to be correct, the reasoning process is not fully grounded in retrieved evidence, revealing a hidden retrieval failure\. Such behaviors may also undermine reinforcement learning optimization, as correct predictions supported by unreliable reasoning can receive positive rewards, potentially reinforcing suboptimal retrieval patterns\.

In contrast,SOARcontinues retrieval when evidence remains incomplete, encouraging evidence\-grounded reasoning instead of correctness by chance\. This behavior improves the reliability of knowledge\-intensive reasoning beyond final\-answer accuracy\.

### I\.2Source\-Aware Reasoning Improves Deep Evidence Retrieval

To further investigate the effectiveness of SOAR in complex scenarios, we conduct a case study on a multi\-hop factual question:"Which film was released earlier, Ek Ladka Ek Ladki or The Original Latin Kings Of Comedy?"This task is challenging as it requires the model to aggregate and compare temporal data from disparate sources\.

Table[16](https://arxiv.org/html/2610.13157#A11.T16)highlights a typical failure mode ofSearch\-r1, where the model resorts to unreliable hallucinations when faced with insufficient external evidence\. The initial search only retrieves information for one film\. Instead of seeking the missing data, the model relies on its internal information, leading to a hallucinated release date \(1978\) and an incorrect final answer\.

In contrast,SOARdemonstrates superiorself\-verification capabilities\. Upon identifying that the retrieved information is incomplete and that its internal knowledge might be unreliable, SOAR proactively initiates a second search targeted at the missing entity\. By synthesizing the confirmed dates from multiple external sources \(1992 vs\. 2002\), it successfully resolves the conflict and derives the correct conclusion\. This case highlights how SOAR effectively mitigates hallucinations in knowledge\-intensive tasks\.

## Appendix JScientific Artifacts

### J\.1Licensing

We use publicly available datasets, models, and software artifacts in accordance with their respective licenses and terms of use\.

### J\.2Intended Use

All external artifacts used in this work are employed strictly in accordance with their intended usage and licensing terms\. Furthermore, any derivatives or findings from this study are designated solely for academic research purposes\.

### J\.3Data Privacy and Safety

All datasets used in this work are widely standard academic benchmarks that have already been vetted for privacy and toxicity\. We manually inspected random samples of the data and confirmed that they do not contain personally identifying information \(PII\) or offensive content\.

### J\.4Documentation of Artifacts

The evaluation benchmarks utilized in our experiments are standard, well\-documented open\-source datasets\. Their text domains predominantly cover open\-domain Wikipedia articles and general\-knowledge text\. All evaluation materials are restricted to standard English, focus purely on factual and linguistic reasoning, and do not involve specific demographic or sensitive user profiles\.

## Appendix KUse Of AI Assistants

We used generative AI tools, including ChatGPT, to assist with identifying grammatical and syntactic issues in the manuscript\.

Table 15:Case study illustrating spurious correctness caused by evidence fabrication\. Search\-r1 reaches the correct answer despite failing to retrieve the key supporting evidence\. Instead of grounding the prediction in retrieved information, the model fills the missing evidence using internal knowledge, producing a correct answer by chance\. This hidden retrieval failure may introduce misleading reward signals during reinforcement learning optimization\.Table 16:Case study comparing Search\-r1 and SOAR on a multi\-hop question\. Search\-r1 prematurely relies on hallucinated internal knowledge, leading to insufficient evidence retrieval and an incorrect answer\. In contrast, SOAR explicitly separates internal and external knowledge sources, detects source inconsistency, and triggers deeper evidence retrieval, enabling accurate reasoning and answer generation\.