CAS: Conformalized Agentic Search via Adaptive Retrieval and Policy Weighting

# Conformalized Agentic Search via Adaptive Retrieval and Policy Weighting
Source: [https://arxiv.org/html/2608.20771](https://arxiv.org/html/2608.20771)
Jiayuan SuAffiliation:State Key Laboratory of CAD&CG, Zhejiang University\.Jian ZhangAffiliation:ZJU\-UIUC Institute, Zhejiang University\.Affiliation:Tencent Inc\.\{zixizhu, 12221038, hongweiwang\}@zju\.edu\.cnyulin@intl\.zju\.edu\.cnmatt\.jiayuan\.su@gmail\.comYu LinHongwei Wang

###### Abstract

Search Agents face a severe reliability crisis during reinforcement learning \(RL\) fine\-tuning\. Heuristic Top\-K retrieval often causes critical evidence loss or noise inclusion, while overconfidence induced by progressive RL leads to hallucinated answers and redundant searches\. To build highly reliable agents, we introduce Conformal Prediction \(CP\) and propose Conformalized Agentic Search \(CAS\)\. This framework establishes reliability guarantees on both the retrieval and training sides: on the retrieval side, an Adaptive Prediction Set \(APS\), a specific CP realization, translates statistical coverage into dynamic document truncation to construct prediction sets that are adaptive in size; on the training side, Adaptive Conformal Inference \(ACI\), a dynamic CP algorithm, dynamically constructs prediction sets with controllable coverage to quantify answer confidence, which is then used to penalize low\-confidence trajectories within the Group Relative Policy Optimization \(GRPO\) objective, ensuring the model learns only from reliable ones\. Experiments across single\-hop and multi\-hop QA datasets demonstrate that our framework significantly improves reasoning accuracy while drastically reducing redundant tool invocations, establishing a highly reliable and efficient agent paradigm\. Our code is available at[https://github\.com/S1llyBird/CAS](https://github.com/S1llyBird/CAS)\.

22footnotetext:Corresponding authors\.![Refer to caption](https://arxiv.org/html/2608.20771v1/figure.png)Figure 1:The CAS framework\.\(A\)APS dynamically sizes the retrieved document set based on local query difficulty\.\(B\)ACI modulates the GRPO policy loss by penalizing low\-confidence trajectories \(NLL\>q1−αt\\mathrm\{NLL\}\>q\_\{1\-\\alpha\_\{t\}\}\) to enforce reliable policy optimization\.## 1Introduction

Large Language Models \(LLMs\) have significantly advanced complex problem\-solving by integrating external knowledge[34](https://arxiv.org/html/2608.20771#bib.bib2);[26](https://arxiv.org/html/2608.20771#bib.bib1)\. While traditional Retrieval\-Augmented Generation \(RAG\) employs a static "retrieve\-then\-generate" paradigm[24](https://arxiv.org/html/2608.20771#bib.bib3), the recent emergence of Agentic Search offers a more dynamic and autonomous approach[17](https://arxiv.org/html/2608.20771#bib.bib4);[14](https://arxiv.org/html/2608.20771#bib.bib5);[51](https://arxiv.org/html/2608.20771#bib.bib18)\. By interleaving internal reasoning steps with external information\-gathering actions within a continuous generation trajectory, the agent autonomously plans when to retrieve and how to integrate newly acquired knowledge[48](https://arxiv.org/html/2608.20771#bib.bib6)\.

However, fine\-tuning these agents via reinforcement learning \(RL\) carries inherent risks of unreliability\. First, on the retrieval side, heuristic Top\-K truncation is inherently unreliable\. Given varying query difficulties, a fixed K inevitably leads to the omission of critical facts or the inclusion of distracting noise[27](https://arxiv.org/html/2608.20771#bib.bib8)\. Second, on the training side, LLMs often exhibit overconfidence as RL progresses[23](https://arxiv.org/html/2608.20771#bib.bib38), leading the model to generate hallucinated responses\. Without effective confidence constraints, Search Agents are easily trapped in a cycle of highly inefficient, redundant tool invocations[49](https://arxiv.org/html/2608.20771#bib.bib7)\.

To address these risks, we adopt Conformal Prediction \(CP\)[3](https://arxiv.org/html/2608.20771#bib.bib9), a principled statistical framework that quantifies model uncertainty with rigorous theoretical guarantees\. Unlike heuristic methods, CP provides strong finite\-sample coverage guarantees: given a user\-specified error rateα\\alpha, it constructs a prediction set that satisfies a target coverage level of at least1−α1\-\\alpha\. These properties make CP a theoretically grounded choice for establishing reliability in autonomous systems\. Building on this, we propose Conformalized Agentic Search \(CAS\), which simultaneously applies CP to both the retrieval and training sides to provide rigorous statistical guarantees[20](https://arxiv.org/html/2608.20771#bib.bib10);[37](https://arxiv.org/html/2608.20771#bib.bib11)\.

On the retrieval side, we implement CP via the Adaptive Prediction Set \(APS\) method[33](https://arxiv.org/html/2608.20771#bib.bib17)\. While ensuring a strict marginal coverage guarantee, APS dynamically adjusts the size of the set of retrieved items based on the model’s "uncertainty" regarding the current input[4](https://arxiv.org/html/2608.20771#bib.bib12)\. On the training side, to mitigate the severe calibration degradation and overconfidence inherent in standard RL, we utilize Adaptive Conformal Inference \(ACI\)[11](https://arxiv.org/html/2608.20771#bib.bib13), a dynamic CP algorithm\. ACI dynamically constructs prediction sets with controllable coverage to quantify answer reliability during training\. We optimize the Group Relative Policy Optimization \(GRPO\)[35](https://arxiv.org/html/2608.20771#bib.bib15)process by penalizing low\-confidence trajectories \(both blind overconfidence and erratic underconfidence\), ensuring the model learns only from highly reliable reasoning paths\.

In summary, our core contributions are threefold:

- •A Reliable Theoretical Framework: We propose CAS, the first framework to introduce CP into the RL fine\-tuning of search agents, providing rigorous statistical guarantees for reasoning and retrieval reliability\.
- •CP Constraints on Both Sides: We implement APS on the retrieval side to construct prediction sets that are adaptive in size, ensuring the inclusion of correct evidence\. Concurrently, we apply ACI on the training side to optimize the GRPO process, mitigating low\-confidence outputs\.
- •Superior Accuracy and Efficiency: Extensive experiments across single\-hop and multi\-hop QA datasets demonstrate that our framework significantly improves reasoning accuracy and training stability while drastically reducing redundant tool invocations, yielding a highly reliable and efficient agent paradigm\.

## 2Conformal Prediction

CP[3](https://arxiv.org/html/2608.20771#bib.bib9)is a principled statistical framework that quantifies uncertainty with rigorous coverage guarantees, regardless of the underlying model or data distribution\. Central to CP is a non\-conformity score functions⁡\(x,y\)s\(x,y\), which measures the “unusualness” of a candidate outputyygiven an inputxx\.

Let\(X,Y\)\(X,Y\)be a sample, whereXXrepresents the input andYYrepresents the output\. Suppose we have a calibration set ofnnsamples, denoted as\(Xi,Yi\)i=1n\(X\_\{i\},Y\_\{i\}\)\_\{i=1\}^\{n\}, and a test sample\(Xt​e​s​t,Yt​e​s​t\)\(X\_\{test\},Y\_\{test\}\)drawn independently and identically \(i\.i\.d\.\) from the same underlying distribution\. Given a user\-specified target error rateα∈\(0,1\)\\alpha\\in\(0,1\), CP computes a quantile thresholdq^\\hat\{q\}corresponding to the⌈\(n\+1\)​\(1−α\)⌉n\\frac\{\\lceil\(n\+1\)\(1\-\\alpha\)\\rceil\}\{n\}empirical quantile of the calibration scores\. It then constructs a prediction set𝒞1−α​\(Xt​e​s​t\)\\mathcal\{C\}\_\{1\-\\alpha\}\(X\_\{test\}\)defined as:

𝒞1−α​\(Xt​e​s​t\)=\{y∈𝒴:s⁡\(Xt​e​s​t,y\)≤q^\}\\mathcal\{C\}\_\{1\-\\alpha\}\(X\_\{test\}\)=\\\{y\\in\\mathcal\{Y\}:s\(X\_\{test\},y\)\\leq\\hat\{q\}\\\}\(1\)Under the i\.i\.d\. assumption, this procedure formally guarantees marginal coverage:P⁡\(Yt​e​s​t∈𝒞1−α​\(Xt​e​s​t\)\)≥1−αP\(Y\_\{test\}\\in\\mathcal\{C\}\_\{1\-\\alpha\}\(X\_\{test\}\)\)\\geq 1\-\\alpha\. See Appendix[B\.1](https://arxiv.org/html/2608.20771#A2.SS1)for the formal proof\.

### 2\.1Adaptive Prediction Set

As a specific realization of the CP framework for classification and generative tasks, APS[33](https://arxiv.org/html/2608.20771#bib.bib17)constructs prediction sets by defining a specialized non\-conformity score\. Specifically, given sorted predictive probabilitiesπ\(1\)​\(x\)≥⋯≥π\(n\)​\(x\)\\pi\_\{\(1\)\}\(x\)\\geq\\cdot\\cdot\\cdot\\geq\\pi\_\{\(n\)\}\(x\), APS defines the non\-conformity scores⁡\(x,y\)s\(x,y\)as the cumulative mass up to the true labelyy:s=∑j=1L⁡\(y\)π\(j\)​\(x\)s=\\sum\_\{j=1\}^\{L\(y\)\}\\pi\_\{\(j\)\}\(x\), whereL⁡\(y\)L\(y\)denotes the rank ofyy\. During the CP inference phase, APS identifies the minimum indexkkto form the prediction set such that the cumulative probability exceeds the calibrated thresholdq^\\hat\{q\}:

∑i=1kπ\(i\)​\(x\)≥q^\\sum\_\{i=1\}^\{k\}\\pi\_\{\(i\)\}\(x\)\\geq\\hat\{q\}\(2\)This mechanism adaptively yields compact sets for confident inputs and expanded sets for ambiguous ones, strictly maintaining the1−α1\-\\alphaCP coverage guarantee while asymptotically approximating conditional coverage\. See Appendix[B\.2](https://arxiv.org/html/2608.20771#A2.SS2)for the formal proof and discussions on its conditional coverage\.

### 2\.2Adaptive Conformal Inference

While the standard CP framework fundamentally relies on the exchangeability \(i\.i\.d\.\) assumption, ACI[11](https://arxiv.org/html/2608.20771#bib.bib13)is a dynamic extension designed to handle data streams where the underlying data distribution may change over time\. Instead of maintaining a static target error rate, ACI introduces a time\-varying error parameterαt\\alpha\_\{t\}\.

At each time steptt, we observe a test point\(Xt,Yt\)\(X\_\{t\},Y\_\{t\}\), whereXtX\_\{t\}is the input andYtY\_\{t\}is the true response\. The algorithm evaluates whetherYtY\_\{t\}was contained within the previous prediction set via the empirical miscoverage indicator:

e​r​rt:=\{1,if​Yt∉𝒞t​\(αt\)0,otherwiseerr\_\{t\}:=\\begin\{cases\}1,&\\text\{if \}Y\_\{t\}\\notin\\mathcal\{C\}\_\{t\}\(\\alpha\_\{t\}\)\\\\ 0,&\\text\{otherwise\}\\end\{cases\}\(3\)where𝒞t​\(αt\):=\{y∈𝒴:s⁡\(Xt,y\)≤Q^t​\(1−αt\)\}\\mathcal\{C\}\_\{t\}\(\\alpha\_\{t\}\):=\\\{y\\in\\mathcal\{Y\}:s\(X\_\{t\},y\)\\leq\\hat\{Q\}\_\{t\}\(1\-\\alpha\_\{t\}\)\\\}is the dynamic prediction set, andQ^t​\(⋅\)\\hat\{Q\}\_\{t\}\(\\cdot\)is the empirical quantile function\.

Given a long\-term target error rateρ\\rhoand a step sizeγ\>0\\gamma\>0, ACI updates the error parameter via a simple online rule:

αt\+1=αt\+γ⁡\(ρ−e​r​rt\)\\alpha\_\{t\+1\}=\\alpha\_\{t\}\+\\gamma\(\\rho\-err\_\{t\}\)\(4\)This recursive mechanism acts as a feedback loop: miscoverage \(e​r​rt=1err\_\{t\}=1\) decreasesαt\\alpha\_\{t\}, thereby expanding the subsequent prediction set to be more conservative\. Conversely, success \(e​r​rt=0err\_\{t\}=0\) increasesαt\\alpha\_\{t\}, tightening the set\. By continuously adaptingαt\\alpha\_\{t\}, ACI preserves valid uncertainty quantification even when the data distribution changes over time\. The formal proof of this dynamic guarantee is provided in Appendix[B\.3](https://arxiv.org/html/2608.20771#A2.SS3)\.

## 3Methodology

We present the CAS framework \(Figure[1](https://arxiv.org/html/2608.20771#S0.F1)\) to reliably bridge LLMs’ internal reasoning with external retrieval\. We introduce two synergistic modules for statistical reliability: APS to bound dynamic retrieval uncertainty, and ACI to penalize low\-confidence trajectories during policy optimization\.

### 3\.1Overview of Agentic Search

Our policy modelπθ​\(y\|x\)\\pi\_\{\\theta\}\(y\|x\)employs a strict generative grammar to interleave internal reasoning with external actions\. Given an inputxx, the model initiates reasoning within<think\>\.\.\.</think\>tags\.Upon reaching a knowledge boundary, it emits a queryqqenclosed in<search\>\.\.\.</search\>tags,, halting generation to invoke an external search engine𝒮\\mathcal\{S\}\. Crucially, rather than using a fixed top\-kk, the raw retrieved evidenceD=𝒮⁡\(q\)D=\\mathcal\{S\}\(q\)is dynamically truncated into a reliable subsetDAPSD\_\{\\text\{APS\}\}via an APS\. This subset is then wrapped in<information\>\.\.\.</information\>tags and appended to the context\. This generation\-retrieval cycle repeats until the model outputs its final predictionapreda\_\{\\text\{pred\}\}inside<answer\>\.\.\.</answer\>tags\. The complete prompt template is provided in Table[5](https://arxiv.org/html/2608.20771#A5.T5)\.

### 3\.2Retrieval Side via APS

Traditional tool\-use frameworks typically append a fixed number of top\-kkresults to the context, which often injects redundant noise or truncates critical information\. To rigorously bound the uncertainty of external evidence, we frame our dynamic retrieval mechanism within the general CP paradigm \(Eq\. \([1](https://arxiv.org/html/2608.20771#S2.E1)\)\)\. However, while standard CP methods guarantee marginal coverage across the data distribution, they fail to guarantee conditional coverage—often failing to adapt to the specific difficulty of a given input\.

To heuristically approximate conditional coverage, we specifyαA​P​S\\alpha\_\{APS\}and construct a reliable document subset by implementing the APS detailed in Section[2\.1](https://arxiv.org/html/2608.20771#S2.SS1), thereby obtaining the calibrated thresholdq^A​P​S\\hat\{q\}\_\{APS\}\. Given a queryqq, the search engine𝒮\\mathcal\{S\}returns an initial candidate setD=\{d1,d2,…,dn\}D=\\\{d\_\{1\},d\_\{2\},\\dots,d\_\{n\}\\\}accompanied by their raw retrieval scores\. We normalize these scores using a softmax operation to obtain a relevance probabilityp⁡\(di\|q\)p\(d\_\{i\}\|q\)for each document\. Following the rigorous APS inference procedure, our system identifies the truncation indexkkby accumulating these probabilities until the mass first exceeds the calibrated thresholdq^APS\\hat\{q\}\_\{\\text\{APS\}\}\. By dynamically adjusting to the conditional probability distribution ofqq, this process yields a statistically guaranteed subsetDAPS=\{d1,…,dk\}D\_\{\\text\{APS\}\}=\\\{d\_\{1\},\\dots,d\_\{k\}\\\}\. Crucially, this provides a statistical guarantee that ensures a1−αA​P​S1\-\\alpha\_\{APS\}coverage rate while adapting the context length to the difficulty of q\.

### 3\.3Reward Design

In our RL framework, we train the policyπθ\\pi\_\{\\theta\}using a rule\-based rewardr⁡\(x,y\)=ra​c​c\+rf​m​tr\(x,y\)=r\_\{acc\}\+r\_\{fmt\}\. We definera​c​c=E​M​\(ap​r​e​d,ag​o​l​d\)∈\{0,1\}r\_\{acc\}=EM\(a\_\{pred\},a\_\{gold\}\)\\in\\\{0,1\\\}as the exact match indicator\. To enforce structural integrity,rf​m​t​\(y\)r\_\{fmt\}\(y\)incorporates two Boolean indicators,𝕀valid\\mathbb\{I\}\_\{\\text\{valid\}\}and𝕀ans\\mathbb\{I\}\_\{\\text\{ans\}\}, representing strict grammatical correctness and the successful generation of the<answer\>boundary, respectively\. With a scaling factorγ=0\.2\\gamma=0\.2, the format reward is formulated as:

rfmt\(y\)=γ⋅\[−racc\(1−𝕀valid\)\+\(1−racc\)\(𝕀valid\+12𝕀ans\(1−𝕀valid\)\)\]r\_\{\\text\{fmt\}\}\(y\)=\\gamma\\cdot\\Big\[\-r\_\{\\text\{acc\}\}\(1\-\\mathbb\{I\}\_\{\\text\{valid\}\}\)\\\\ \+\(1\-r\_\{\\text\{acc\}\}\)\\left\(\\mathbb\{I\}\_\{\\text\{valid\}\}\+\\frac\{1\}\{2\}\\mathbb\{I\}\_\{\\text\{ans\}\}\(1\-\\mathbb\{I\}\_\{\\text\{valid\}\}\)\\right\)\\Big\]\(5\)
This formulation rewards the correct format: when the answer is correct \(racc=1r\_\{\\text\{acc\}\}=1\), it strictly applies a−γ\-\\gammapenalty for format violations to prevent reward hacking\. Conversely, when the answer is incorrect \(racc=0r\_\{\\text\{acc\}\}=0\), it provides a dense intermediate signal \(γ\\gammafor full validity, or12​γ\\frac\{1\}\{2\}\\gammafor partial structural effort\) to guide the model toward using the correct format\.

### 3\.4Training Side via ACI

Standard CP relies on the strict exchangeability \(i\.i\.d\.\) assumption\. However, during RL, the policyπθ\\pi\_\{\\theta\}continuously evolves, and the prevalent use of binary rewards often leads to model overconfidence\. To mitigate the redundant invocations and hallucinated outputs caused by this overconfidence, we employ ACI\.

In our framework, at each RL iterationtt, for theii\-th sampled trajectoryyiy\_\{i\}given inputxtx\_\{t\}, we define its non\-conformity scores⁡\(xt,yi\)s\(x\_\{t\},y\_\{i\}\)as the Negative Log\-Likelihood \(NLL\)[31](https://arxiv.org/html/2608.20771#bib.bib19)of the generated tokens strictly within the<answer\>\.\.\.</answer\>tags\. Letℳi\\mathcal\{M\}\_\{i\}denote the set of token indices corresponding to these final answer tokens\. The score is formulated as:

s⁡\(xt,yi\)=∑j∈ℳi\(−log⁡πθ​\(yj\|y<j,xt\)\)\|ℳi\|s\(x\_\{t\},y\_\{i\}\)=\\frac\{\\sum\_\{j\\in\\mathcal\{M\}\_\{i\}\}\(\-\\log\\pi\_\{\\theta\}\(y\_\{j\}\|y\_\{<j\},x\_\{t\}\)\)\}\{\|\\mathcal\{M\}\_\{i\}\|\}\(6\)
Given a pre\-specified target error rateρ\\rho, instead of using the standard update rule, we employ a smoothed ACI update mechanism over previously observed data to mitigate local variations in the error rate[11](https://arxiv.org/html/2608.20771#bib.bib13)\. Specifically, we update the error rateαt\\alpha\_\{t\}by evaluating the recent empirical miscoverage frequency using an exponentially weighted moving average of past errors:

αt\+1=αt\+γ⁡\(ρ−∑s=1tvs​e​r​rs\)\\alpha\_\{t\+1\}=\\alpha\_\{t\}\+\\gamma\\left\(\\rho\-\\sum\_\{s=1\}^\{t\}v\_\{s\}err\_\{s\}\\right\)\(7\)where\{vs\}1≤s≤t\\\{v\_\{s\}\\\}\_\{1\\leq s\\leq t\}is a sequence of increasing weights such that∑s=1tvs=1\\sum\_\{s=1\}^\{t\}v\_\{s\}=1\. In practice, we define the temporal weights with a smoothing factor of0\.950\.95as

vs:=0\.95t−s∑s′=1t0\.95t−s′\.v\_\{s\}:=\\frac\{0\.95^\{t\-s\}\}\{\\sum\_\{s^\{\\prime\}=1\}^\{t\}0\.95^\{t\-s^\{\\prime\}\}\}\.This approach effectively produces smoother trajectories forαt\\alpha\_\{t\}with less local variation\. The updatedαt\\alpha\_\{t\}is then used to dynamically adjust the quantile thresholdq^t=Q^t​\(1−αt\)\\hat\{q\}\_\{t\}=\\hat\{Q\}\_\{t\}\(1\-\\alpha\_\{t\}\)\. Specifically, the empirical quantile functionQ^t​\(⋅\)\\hat\{Q\}\_\{t\}\(\\cdot\)is evaluated over a rolling calibration window of past scores from iterationr=max⁡\(1,t−2000\)r=\\max\(1,t\-2000\)tot−1t\-1\. This thresholdq^t\\hat\{q\}\_\{t\}is subsequently employed to partition low\-confidence trajectories\.

We employ GRPO[35](https://arxiv.org/html/2608.20771#bib.bib15)for the RL training\. For a given inputxtx\_\{t\}at iterationtt, the policy samples a group ofGGtrajectories\{y1,y2,…,yG\}\\\{y\_\{1\},y\_\{2\},\\dots,y\_\{G\}\\\}\. GRPO optimizes the policy by computing the relative advantageAiA\_\{i\}for each trajectoryyiy\_\{i\}, which is obtained by normalizing its rewardRiR\_\{i\}within the group:Ai=Ri−mean​\(𝐑\)std​\(𝐑\)A\_\{i\}=\\frac\{R\_\{i\}\-\\text\{mean\}\(\\mathbf\{R\}\)\}\{\\text\{std\}\(\\mathbf\{R\}\)\}\.

Concurrently, to penalize low\-confidence sequences generated during training, we introduce a discount factorη∈\(0,1\)\\eta\\in\(0,1\)for samples falling into the low\-confidence set\. Accordingly, we minimize the ACI\-guided GRPO loss function as follows:

ℒG​R​P​O\(θ\)=𝔼\[1G∑i=1GωiLic​l​i​p\(θ\)−βDK​L\(πθ\|\|πr​e​f\)\]\\mathcal\{L\}\_\{GRPO\}\(\\theta\)=\\mathbb\{E\}\\left\[\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}\\omega\_\{i\}L\_\{i\}^\{clip\}\(\\theta\)\-\\beta D\_\{KL\}\(\\pi\_\{\\theta\}\|\|\\pi\_\{ref\}\)\\right\]

\(8\)whereβ​DK​L\\beta D\_\{KL\}is the KL divergence penalty against the reference modelπr​e​f\\pi\_\{ref\}, andLic​l​i​p​\(θ\)L\_\{i\}^\{clip\}\(\\theta\)is the standard GRPO clipped objective function driven by the advantageAiA\_\{i\}\. The dynamic confidence\-based weightωi\\omega\_\{i\}is defined as:

ωi=\{1,if​s​\(xt,yi\)≤q^tη,otherwise\\omega\_\{i\}=\\begin\{cases\}1,&\\text\{if \}s\(x\_\{t\},y\_\{i\}\)\\leq\\hat\{q\}\_\{t\}\\\\ \\eta,&\\text\{otherwise\}\\end\{cases\}\(9\)
Crucially, this weighting mechanism synergizes with the RL advantageAiA\_\{i\}to provide a dual\-constraint on model reliability\. For unconfident lucky guesses \(s⁡\(xt,yi\)\>q^ts\(x\_\{t\},y\_\{i\}\)\>\\hat\{q\}\_\{t\}withAi\>0A\_\{i\}\>0\), the positive reinforcement is discounted byη\\eta, preventing the model from learning to guess\. Conversely, if the model is confidently incorrect \(s⁡\(xt,yi\)≤q^ts\(x\_\{t\},y\_\{i\}\)\\leq\\hat\{q\}\_\{t\}but yieldingAi<0A\_\{i\}<0\), the full weight \(ωi=1\\omega\_\{i\}=1\) ensures the model receives the maximum penalty\.

To effectively penalize low\-confidence samples during early training while preventing over\-penalization of relatively high\-confidence trajectories classified as low\-confidence after the model converges, we set the discount factor toη=0\.5\\eta=0\.5\. The complete procedure of CAS is formally presented in Algorithm[1](https://arxiv.org/html/2608.20771#algorithm1)\.

## 4Experiments

Table 1:The main results on seven datasets\.†/⋆\\dagger/\\starrepresents in\-domain/out\-of\-domain datasets\. The best and second best performances are set as bold and underlined, respectively\.Table 2:Performance improvements of our method compared to Search\-R1 and Search\-R2 on General QA \(single\-hop\) and Multi\-Hop QA\.Δ\\Deltadenotes the absolute performance gain\.### 4\.1Experimental Setup

#### Datasets\.

To comprehensively evaluate CAS, we conduct experiments on seven diverse open\-domain Question Answering \(QA\) datasets covering both single\-hop and complex reasoning capabilities: NQ[21](https://arxiv.org/html/2608.20771#bib.bib20), TriviaQA[18](https://arxiv.org/html/2608.20771#bib.bib21), PopQA[29](https://arxiv.org/html/2608.20771#bib.bib22), HotpotQA[47](https://arxiv.org/html/2608.20771#bib.bib23), 2WikiMultihopQA \(2Wiki\)[15](https://arxiv.org/html/2608.20771#bib.bib24), MuSiQue[39](https://arxiv.org/html/2608.20771#bib.bib25), and Bamboogle[30](https://arxiv.org/html/2608.20771#bib.bib26)\. For training our RL framework, we construct a mixed training corpus using the training splits of NQ and HotpotQA\.

#### Baselines\.

We compare our method against a comprehensive suite of competitive baselines, which can be logically categorized into four groups: inference without retrieval, including Direct Inference and CoT reasoning[43](https://arxiv.org/html/2608.20771#bib.bib27); inference with retrieval, comprising standard RAG[24](https://arxiv.org/html/2608.20771#bib.bib3), IRCoT[40](https://arxiv.org/html/2608.20771#bib.bib28), and Search\-o1[25](https://arxiv.org/html/2608.20771#bib.bib29); fine\-tuning based methods, which involve Supervised Fine\-Tuning \(SFT\)[6](https://arxiv.org/html/2608.20771#bib.bib30), RL\-based fine\-tuning without search \(R1\)[12](https://arxiv.org/html/2608.20771#bib.bib31), and rejection sampling with a search engine[1](https://arxiv.org/html/2608.20771#bib.bib32); and finally, our reference backbone, Search\-R1[17](https://arxiv.org/html/2608.20771#bib.bib4), along with Search\-R2[14](https://arxiv.org/html/2608.20771#bib.bib5)\.

#### Implementation Details\.

We initialize our policy model using Qwen2\.5\-3B[32](https://arxiv.org/html/2608.20771#bib.bib35)and Qwen3\-8B[46](https://arxiv.org/html/2608.20771#bib.bib36)\. For the retrieval module, we utilize the dense retriever E5[42](https://arxiv.org/html/2608.20771#bib.bib33), paired with the 2018 Wikipedia dump[19](https://arxiv.org/html/2608.20771#bib.bib34)as the external knowledge base\. During the GRPO training phase, we set the group rollout size toG=5G=5and sample 512 prompts per training step\. To prevent infinite generation, the maximum number of assistant\-search interaction rounds is capped at 4\. The learning rate is set to1×10−61\\times 10^\{\-6\}\. For the ACI module, the step size is empirically set toγ=0\.005\\gamma=0\.005\. All models are evaluated using the Exact Match \(EM\) metric\. We provide more details in Appendix[C](https://arxiv.org/html/2608.20771#A3)\.

#### Conformal Settings\.

We setαAPS=0\.2\\alpha\_\{\\text\{APS\}\}=0\.2and the target error rateρ=0\.25\\rho=0\.25for ACI\. The ACI calibration set𝒞\\mathcal\{C\}comprises 150 randomly sampled queries from a mixture of NQ and HotpotQA training splits\. For ACI, we evaluate the untrained backbone on𝒞\\mathcal\{C\}to compute the initial non\-conformity scores \(Eq\. \([6](https://arxiv.org/html/2608.20771#S3.E6)\)\), establishing the base empirical quantileqα0q\_\{\\alpha\_\{0\}\}to bootstrap the dynamic tracking process\. For APS calibration, we construct a distinct calibration set𝒞APS\\mathcal\{C\}\_\{\\text\{APS\}\}\. We utilize DeepSeek\-V3\.2[8](https://arxiv.org/html/2608.20771#bib.bib37)to decompose multi\-hop queries from the original 150 samples\. These decomposed sub\-queries, alongside the original single\-hop questions, form𝒞APS\\mathcal\{C\}\_\{\\text\{APS\}\}, which totals 239 queries\. Given the targetαAPS=0\.2\\alpha\_\{\\text\{APS\}\}=0\.2, this sample size provides a statistical error margin ofϵ=0\.026\\epsilon=0\.026for the actual coverage[3](https://arxiv.org/html/2608.20771#bib.bib9)\. DeepSeek\-V3\.2 then acts as the judge to locate the ground\-truth documents within𝒞APS\\mathcal\{C\}\_\{\\text\{APS\}\}, yielding the calibrated thresholdq^APS\\hat\{q\}\_\{\\text\{APS\}\}as formulated in Section[3\.2](https://arxiv.org/html/2608.20771#S3.SS2)\.

### 4\.2Main Results

Table[1](https://arxiv.org/html/2608.20771#S4.T1)and Table[2](https://arxiv.org/html/2608.20771#S4.T2)present the comprehensive evaluation results of our method against all baselines across the seven datasets\. We summarize the key findings as follows:

CAS achieves superior performance across all evaluated settings\. Specifically, on the Qwen3\-8B backbone, our method achieves the highest overall average score of 0\.464, outperforming the strong baseline Search\-R2 \(0\.446\) and substantially surpassing Search\-R1 \(0\.400\)\. A similar trend is observed on the Qwen2\.5\-3B\-Instruct backbone, where our method achieves an average score of 0\.401, improving upon Search\-R1 by an absolute margin of \+0\.065\. This consistent superiority across different model scales highlights the generalizability and robustness of CAS\.11footnotetext:As Search\-R2 is closed\-source, we are unable to evaluate it on Qwen2\.5\-3B\-Instruct\.

Our approach demonstrates exceptional performance across both complex multi\-step and straightforward single\-hop scenarios\. As shown in Table[2](https://arxiv.org/html/2608.20771#S4.T2), on Multi\-Hop QA datasets, our method yields massive gains, outperforming Search\-R1 by \+0\.079 \(3B\) and \+0\.078 \(8B\), and surpassing the highly optimized Search\-R2 by \+0\.032\. On General QA \(single\-hop\) tasks, our models also achieve highly competitive accuracy, with the 8B backbone significantly outperforming Search\-R1 \(\+0\.045\) and slightly edging out Search\-R2 \(\+0\.001\)\. This demonstrates the improvements of our framework: APS provides a retrieval set with marginal coverage and approximate conditional coverage to prevent the model from encountering hallucinations due to excessive context in simple queries or missing answers in complex ones; meanwhile, ACI effectively ensures high\-confidence model outputs, preventing hallucinations and redundant tool invocations\. Further experimental analysis regarding the Qwen3\-8B backbone is deferred to Appendix[E](https://arxiv.org/html/2608.20771#A5)\.

Table 3:Ablation study on Qwen2\.5\-3B\-Instruct\. The table presents the unablated framework, component\-wise ablations, and sensitivity analyses forρ\\rhoandαAPS\\alpha\_\{\\text\{APS\}\}\.Full results across all individual datasets are detailed in Table[8](https://arxiv.org/html/2608.20771#A6.T8)\.\(a\)Average Search Calls\(b\)Ratio of Low\-Confidence Samples\(c\)Adaptive Target Error Rate \(αt\\alpha\_\{t\}\)
Figure 2:Training dynamics and sensitivity analyses on the Qwen2\.5\-3B\-Instruct backbone\.\(a\)Evolution of average search calls during training across different ablation configurations\.\(b\)The ratio of low\-confidence samples penalized by the ACI weight under varying target error rates \(ρ\\rho\)\.\(c\)The dynamic adaptation of the target error rate \(αt\\alpha\_\{t\}\), demonstrating stable convergence to the presetρ\\rhovalues\.
### 4\.3Ablation Study

To evaluate the individual contributions of our proposed modules, we conducted a component\-wise ablation study on the Qwen2\.5\-3B\-Instruct backbone\. For the configuration where the APS is disabled \(\-APS\), the retrieval mechanism falls back to a fixed top\-kk\(k=3k=3\) setting\. As shown in Table[3](https://arxiv.org/html/2608.20771#S4.T3), removing either component leads to a notable degradation in both General QA and Multi\-Hop QA tasks\.

#### Impact of the ACI Weight\.

Removing the ACI weight \(\-ACI\) decreases the overall score from 0\.401 to 0\.384\. This decline is intrinsically linked to the model’s search behavior\. As illustrated in the tool usage trajectories \(Figure[2](https://arxiv.org/html/2608.20771#S4.F2)\(a\)\), the baseline and the \-ACI variant exhibit significantly higher and more fluctuating tool calls\. Without confidence constraints, blind overconfidence causes the model to hallucinate, initiating searches that deviate from the target question\. Notably, some trajectories, due to a lack of confidence, conversely resort to secondary searches to verify answers\. In summary, through the ACI constraint, the model avoids not only overconfidence but also blind underconfidence, thereby maintaining a stable and efficient search frequency \(as denoted by the default trajectory\)\.

#### Impact of the Adaptive Prediction Set\.

Disabling APS \(\-APS\) drops accuracy to 0\.389 by restricting retrieval to a fixed\-length context\. This rigid setup degrades performance via two paths: in simple queries, fixed top\-kkretrieval introduces noise through redundant documents; in complex multi\-hop queries, the static window often misses critical facts\. Notably, the resulting information scarcity in complex scenarios forces the model to issue additional tool calls to compensate, even with the ACI weight active\. Consequently, its tool usage frequency falls between our full framework and the baseline \(Figure[2\(a\)](https://arxiv.org/html/2608.20771#S4.F2.sf1)\)\. This dynamic corroborates the complementarity of the two modules: APS provides an adaptive, noise\-free context in a single step, while the ACI weight suppresses unnecessary exploratory searches\.

### 4\.4Sensitivity Analysis

To verify the robustness and controllability of our framework, we conduct a sensitivity analysis on the ACI target error rateρ∈\{0\.1,0\.25,0\.4\}\\rho\\in\\\{0\.1,0\.25,0\.4\\\}and the APS significance levelαAPS∈\{0\.05,0\.2,0\.35\}\\alpha\_\{\\text\{APS\}\}\\in\\\{0\.05,0\.2,0\.35\\\}\. All experiments in this section are performed on the Qwen2\.5\-3B\-Instruct backbone, with calibration set configurations consistent with the Conformal Settings\. The performance results are summarized in the bottom sections of Table[3](https://arxiv.org/html/2608.20771#S4.T3)\.

#### ACI Weight under Differentρ\\rho\.

Figures[2\(b\)](https://arxiv.org/html/2608.20771#S4.F2.sf2)and[2\(c\)](https://arxiv.org/html/2608.20771#S4.F2.sf3)illustrate the dynamic characteristics of the ACI mechanism during RL fine\-tuning\. The ACI weight effectively maintains the ratio of low\-confidence samples within expected ranges and ensures that the dynamically adjustedαt\\alpha\_\{t\}closely tracks the target error rateρ\\rho\. Notably, a pronounced spike is observed in the ratio of low\-confidence samples at the very first step \(figure[2\(b\)](https://arxiv.org/html/2608.20771#S4.F2.sf2)\)\. This phenomenon is directly attributed to the surge of trajectories as the policyπθ\\pi\_\{\\theta\}begins to update, introducing highly non\-i\.i\.d\. data into the stream\. The rapid stabilization ofαt\\alpha\_\{t\}following this shock demonstrates ACI’s robust adaptability, highlighting the fundamental inadequacy of Static CP in dynamic RL environments\.

Furthermore, as shown in Table[3](https://arxiv.org/html/2608.20771#S4.T3), a strict target \(ρ=0\.4\\rho=0\.4\) classifies nearly 40% of the reasoning trajectories as low\-confidence\. While enforcing rigorous quality constraints, this over\-penalization severely dilutes the RL reward signals, diminishing training efficiency\. Conversely, a relaxed target \(ρ=0\.1\\rho=0\.1\) applies the ACI weight to only 10% of the samples\. With such lenient filtering, the performance degenerates toward the unconstrained baseline due to insufficient confidence guidance\.

Table 4:Average number of retrieved documents under different APS significance levels \(αAPS\\alpha\_\{\\text\{APS\}\}\)\.
#### APS Retrieval under DifferentαAPS\\alpha\_\{\\text\{APS\}\}\.

The significance levelαAPS\\alpha\_\{\\text\{APS\}\}dictates the aggressiveness of the dynamic context truncation\. Table[4](https://arxiv.org/html/2608.20771#S4.T4)presents the average number of retrieved documents under differentαAPS\\alpha\_\{\\text\{APS\}\}settings\. A high\-guarantee setting \(αAPS=0\.05\\alpha\_\{\\text\{APS\}\}=0\.05\) yields an average of 4\.8 documents, ensuring a 95% marginal coverage\. While this extensive context significantly benefits single\-hop General QA by minimizing the risk of omitting critical evidence, the excessive information introduces substantial noise, which severely impairs the reasoning quality in complex Multi\-Hop QA \(see Table[3](https://arxiv.org/html/2608.20771#S4.T3)\)\. In contrast, a low\-guarantee setting \(αAPS=0\.35\\alpha\_\{\\text\{APS\}\}=0\.35\) returns only 2\.4 documents on average\. This aggressive truncation fails to provide sufficient supporting facts, resulting in suboptimal performance across both tasks\. Consequently, our default configuration \(αAPS=0\.20\\alpha\_\{\\text\{APS\}\}=0\.20\) strikes the optimal balance between comprehensive information retrieval and effective noise reduction\.

## 5Related Works

### 5\.1Retrieval in LLMs

Traditional RAG[24](https://arxiv.org/html/2608.20771#bib.bib3);[10](https://arxiv.org/html/2608.20771#bib.bib14)significantly expands the knowledge boundaries of LLMs by prepending retrieved external documents to the input context\. With the continuous evolution of RAG, the emergence of frameworks such as Adaptive RAG[16](https://arxiv.org/html/2608.20771#bib.bib40), Search\-o1[25](https://arxiv.org/html/2608.20771#bib.bib29), and SAKI\-RAG[38](https://arxiv.org/html/2608.20771#bib.bib51)has highlighted the inherent challenges of determining when to trigger retrieval in static paradigms\. Concurrently, approaches that integrate retrieval with Reinforcement Learning[17](https://arxiv.org/html/2608.20771#bib.bib4);[14](https://arxiv.org/html/2608.20771#bib.bib5);[51](https://arxiv.org/html/2608.20771#bib.bib18);[36](https://arxiv.org/html/2608.20771#bib.bib43), have been introduced\. However, these methods universally rely on fixed Top\-K truncation\. This static constraint fails to guarantee the marginal coverage of the retrieved knowledge\.

### 5\.2Reinforcement Learning for Agents

Reinforcement Learning has fundamentally transformed the capabilities of LLMs, evolving them from passive generators into autonomous agents[28](https://arxiv.org/html/2608.20771#bib.bib46), such as SWE\-agents[50](https://arxiv.org/html/2608.20771#bib.bib41);[44](https://arxiv.org/html/2608.20771#bib.bib44), Web Agents[9](https://arxiv.org/html/2608.20771#bib.bib42);[13](https://arxiv.org/html/2608.20771#bib.bib45), and Search Agents[17](https://arxiv.org/html/2608.20771#bib.bib4);[51](https://arxiv.org/html/2608.20771#bib.bib18);[14](https://arxiv.org/html/2608.20771#bib.bib5);[36](https://arxiv.org/html/2608.20771#bib.bib43)\. However, they universally face the problem of overconfidence driven by sparse, binary rewards, which subsequently leads to hallucination issues[23](https://arxiv.org/html/2608.20771#bib.bib38)\. Recent work proposes training models to explicitly verbalize their confidence scores alongside their answers[7](https://arxiv.org/html/2608.20771#bib.bib39)\. Yet, this approach struggles in Search Agents, as the discontinuous generation process caused by continuous tool invocations prevents the model from explicitly expressing its confidence\.

### 5\.3Conformal Prediction and Uncertainty Quantification

CP[41](https://arxiv.org/html/2608.20771#bib.bib50)offers highly reliable marginal coverage guarantees without distributional assumptions\. Due to its theoretical rigor, CP has been widely applied in traditional classification and detection tasks[45](https://arxiv.org/html/2608.20771#bib.bib48);[2](https://arxiv.org/html/2608.20771#bib.bib49)\. Recent works have integrated CP into LLMs[20](https://arxiv.org/html/2608.20771#bib.bib10);[31](https://arxiv.org/html/2608.20771#bib.bib19);[37](https://arxiv.org/html/2608.20771#bib.bib11)and further into RL fine\-tuning for robust alignment[5](https://arxiv.org/html/2608.20771#bib.bib16)\. However, while CP guarantees marginal coverage, it fails to guarantee conditional coverage, exhibiting a lack of adaptability when faced with complex queries[33](https://arxiv.org/html/2608.20771#bib.bib17)\. Furthermore, CP methods fundamentally rely on a strict independent and identically distributed \(i\.i\.d\.\) assumption, which is inherently violated in reinforcement learning where the model and policy are continuously updating\. Therefore, breaking these constraints is essential for reliable agentic search\.

## 6Conclusion

We propose CAS, a framework that integrates Conformal Prediction to resolve the reliability crisis in RL\-trained search agents\. The synergy between APS and ACI ensures reliable document retrieval and mitigates model overconfidence\. Empirically, CAS significantly enhances reasoning accuracy and reduces redundant tool invocations\. By balancing theoretical rigor with practical performance, this work establishes a principled foundation for future reliable autonomous agents\.

## 7Limitations

Although CAS demonstrates significant potential in improving the accuracy and efficiency of Search Agents, several limitations remain\. First, our empirical validation is primarily focused on general open\-domain Question Answering \(QA\) tasks\. While CAS provides robust statistical guarantees within these general information\-seeking contexts, its applicability in highly specialized professional domains remains unexplored\. Second, the framework relies heavily on a strong external teacher model \(e\.g\., DeepSeek\-V3\.2\) to construct the calibration set for the APS by decomposing queries and acting as a relevance judge\. Finally, CAS primarily focuses on outcome reliability without extending statistical guarantees to the intermediate reasoning process\. Future research should explore verifiable process reliability\.

## Acknowledgments

This work was supported by the National Key Research and Development Program of China \(2024YFF0907802 and 2024YFF0907803\) and the National Natural Science Foundation of China \(62276230\)\.

## References

- Ahnet al\.\(2024\)J\. Ahn, R\. Verma, R\. Lou, D\. Liu, R\. Zhang, and W\. YinLarge language models for mathematical reasoning: progresses and challenges\.External Links:2402\.00157,[Link](https://arxiv.org/abs/2402.00157)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px2.p1.1)\.
- andéolet al\.\(2025\)L\. andéol, L\. Mossina, A\. Mazoyer, and S\. GerchinovitzConformal object detection by sequential risk control\.External Links:2505\.24038,[Link](https://arxiv.org/abs/2505.24038)Cited by:[§5\.3](https://arxiv.org/html/2608.20771#S5.SS3.p1.1)\.
- Angelopoulos and Bates \(2022\)A\. N\. Angelopoulos and S\. BatesA gentle introduction to conformal prediction and distribution\-free uncertainty quantification\.External Links:2107\.07511,[Link](https://arxiv.org/abs/2107.07511)Cited by:[§B\.1](https://arxiv.org/html/2608.20771#A2.SS1.p1.1),[§1](https://arxiv.org/html/2608.20771#S1.p3.1),[§2](https://arxiv.org/html/2608.20771#S2.p1.1),[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px4.p1.1)\.
- Chakrabortyet al\.\(2026\)D\. Chakraborty, E\. Yang, D\. Khashabi, D\. Lawrie, and K\. DuhPrincipled context engineering for rag: statistical guarantees via conformal prediction\.InAdvances in Information Retrieval,pp\. 537–546\.External Links:ISBN 9783032213006,ISSN 1611\-3349,[Link](http://dx.doi.org/10.1007/978-3-032-21300-6_45),[Document](https://dx.doi.org/10.1007/978-3-032-21300-6%5F45)Cited by:[§1](https://arxiv.org/html/2608.20771#S1.p4.1)\.
- Chenet al\.\(2026\)T\. Chen, X\. Liu, V\. Nandam, K\. Liou, and H\. WeiConformal feedback alignment: quantifying answer\-level reliability for robust llm alignment\.External Links:2601\.17329,[Link](https://arxiv.org/abs/2601.17329)Cited by:[§5\.3](https://arxiv.org/html/2608.20771#S5.SS3.p1.1)\.
- Chunget al\.\(2022\)H\. W\. Chung, L\. Hou, S\. Longpre, B\. Zoph, Y\. Tay, W\. Fedus, Y\. Li, X\. Wang, M\. Dehghani, S\. Brahma, A\. Webson, S\. S\. Gu, Z\. Dai, M\. Suzgun, X\. Chen, A\. Chowdhery, A\. Castro\-Ros, M\. Pellat, K\. Robinson, D\. Valter, S\. Narang, G\. Mishra, A\. Yu, V\. Zhao, Y\. Huang, A\. Dai, H\. Yu, S\. Petrov, E\. H\. Chi, J\. Dean, J\. Devlin, A\. Roberts, D\. Zhou, Q\. V\. Le, and J\. WeiScaling instruction\-finetuned language models\.External Links:2210\.11416,[Link](https://arxiv.org/abs/2210.11416)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px2.p1.1)\.
- Damaniet al\.\(2025\)M\. Damani, I\. Puri, S\. Slocum, I\. Shenfeld, L\. Choshen, Y\. Kim, and J\. AndreasBeyond binary rewards: training lms to reason about their uncertainty\.External Links:2507\.16806,[Link](https://arxiv.org/abs/2507.16806)Cited by:[§5\.2](https://arxiv.org/html/2608.20771#S5.SS2.p1.1)\.
- DeepSeek\-AIet al\.\(2025\)DeepSeek\-AI, A\. Liu, A\. Mei, B\. Lin, B\. Xue, B\. Wang, B\. Xu, B\. Wu, B\. Zhang, C\. Lin, C\. Dong, C\. Lu, C\. Zhao, C\. Deng, C\. Xu, C\. Ruan, D\. Dai, D\. Guo, D\. Yang, D\. Chen, E\. Li, F\. Zhou, F\. Lin, F\. Dai, G\. Hao, G\. Chen, G\. Li, H\. Zhang, H\. Xu, H\. Li, H\. Liang, H\. Wei, H\. Zhang, H\. Luo, H\. Ji, H\. Ding, H\. Tang, H\. Cao, H\. Gao, H\. Qu, H\. Zeng, J\. Huang, J\. Li, J\. Xu, J\. Hu, J\. Chen, J\. Xiang, J\. Yuan, J\. Cheng, J\. Zhu, J\. Ran, J\. Jiang, J\. Qiu, J\. Li, J\. Song, K\. Dong, K\. Gao, K\. Guan, K\. Huang, K\. Zhou, K\. Huang, K\. Yu, L\. Wang, L\. Zhang, L\. Wang, L\. Zhao, L\. Yin, L\. Guo, L\. Luo, L\. Ma, L\. Wang, L\. Zhang, M\. S\. Di, M\. Y\. Xu, M\. Zhang, M\. Zhang, M\. Tang, M\. Zhou, P\. Huang, P\. Cong, P\. Wang, Q\. Wang, Q\. Zhu, Q\. Li, Q\. Chen, Q\. Du, R\. Xu, R\. Ge, R\. Zhang, R\. Pan, R\. Wang, R\. Yin, R\. Xu, R\. Shen, R\. Zhang, S\. H\. Liu, S\. Lu, S\. Zhou, S\. Chen, S\. Cai, S\. Chen, S\. Hu, S\. Liu, S\. Hu, S\. Ma, S\. Wang, S\. Yu, S\. Zhou, S\. Pan, S\. Zhou, T\. Ni, T\. Yun, T\. Pei, T\. Ye, T\. Yue, W\. Zeng, W\. Liu, W\. Liang, W\. Pang, W\. Luo, W\. Gao, W\. Zhang, X\. Gao, X\. Wang, X\. Bi, X\. Liu, X\. Wang, X\. Chen, X\. Zhang, X\. Nie, X\. Cheng, X\. Liu, X\. Xie, X\. Liu, X\. Yu, X\. Li, X\. Yang, X\. Li, X\. Chen, X\. Su, X\. Pan, X\. Lin, X\. Fu, Y\. Q\. Wang, Y\. Zhang, Y\. Xu, Y\. Ma, Y\. Li, Y\. Li, Y\. Zhao, Y\. Sun, Y\. Wang, Y\. Qian, Y\. Yu, Y\. Zhang, Y\. Ding, Y\. Shi, Y\. Xiong, Y\. He, Y\. Zhou, Y\. Zhong, Y\. Piao, Y\. Wang, Y\. Chen, Y\. Tan, Y\. Wei, Y\. Ma, Y\. Liu, Y\. Yang, Y\. Guo, Y\. Wu, Y\. Wu, Y\. Cheng, Y\. Ou, Y\. Xu, Y\. Wang, Y\. Gong, Y\. Wu, Y\. Zou, Y\. Li, Y\. Xiong, Y\. Luo, Y\. You, Y\. Liu, Y\. Zhou, Z\. F\. Wu, Z\. Z\. Ren, Z\. Zhao, Z\. Ren, Z\. Sha, Z\. Fu, Z\. Xu, Z\. Xie, Z\. Zhang, Z\. Hao, Z\. Gou, Z\. Ma, Z\. Yan, Z\. Shao, Z\. Huang, Z\. Wu, Z\. Li, Z\. Zhang, Z\. Xu, Z\. Wang, Z\. Gu, Z\. Zhu, Z\. Li, Z\. Zhang, Z\. Xie, Z\. Gao, Z\. Pan, Z\. Yao, B\. Feng, H\. Li, J\. L\. Cai, J\. Ni, L\. Xu, M\. Li, N\. Tian, R\. J\. Chen, R\. L\. Jin, S\. S\. Li, S\. Zhou, T\. Sun, X\. Q\. Li, X\. Jin, X\. Shen, X\. Chen, X\. Song, X\. Zhou, Y\. X\. Zhu, Y\. Huang, Y\. Li, Y\. Zheng, Y\. Zhu, Y\. Ma, Z\. Huang, Z\. Xu, Z\. Zhang, D\. Ji, J\. Liang, J\. Guo, J\. Chen, L\. Xia, M\. Wang, M\. Li, P\. Zhang, R\. Chen, S\. Sun, S\. Wu, S\. Ye, T\. Wang, W\. L\. Xiao, W\. An, X\. Wang, X\. Sun, X\. Wang, Y\. Tang, Y\. Zha, Z\. Zhang, Z\. Ju, Z\. Zhang, and Z\. QuDeepSeek\-v3\.2: pushing the frontier of open large language models\.External Links:2512\.02556,[Link](https://arxiv.org/abs/2512.02556)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px4.p1.1)\.
- Dinget al\.\(2026\)H\. Ding, P\. Liu, J\. Wang, Z\. Ji, M\. Cao, R\. Zhang, L\. Ai, E\. Yang, T\. Shi, and L\. YuDynaWeb: model\-based reinforcement learning of web agents\.External Links:2601\.22149,[Link](https://arxiv.org/abs/2601.22149)Cited by:[§5\.2](https://arxiv.org/html/2608.20771#S5.SS2.p1.1)\.
- Gaoet al\.\(2024\)Y\. Gao, Y\. Xiong, X\. Gao, K\. Jia, J\. Pan, Y\. Bi, Y\. Dai, J\. Sun, M\. Wang, and H\. WangRetrieval\-augmented generation for large language models: a survey\.External Links:2312\.10997,[Link](https://arxiv.org/abs/2312.10997)Cited by:[§5\.1](https://arxiv.org/html/2608.20771#S5.SS1.p1.1)\.
- Gibbs and Candès \(2021\)I\. Gibbs and E\. CandèsAdaptive conformal inference under distribution shift\.External Links:2106\.00170,[Link](https://arxiv.org/abs/2106.00170)Cited by:[§B\.3](https://arxiv.org/html/2608.20771#A2.SS3.p1.1),[§B\.3](https://arxiv.org/html/2608.20771#A2.SS3.p2.1),[§B\.3](https://arxiv.org/html/2608.20771#A2.SS3.p4.1.2),[§B\.3](https://arxiv.org/html/2608.20771#A2.SS3.p7.1.2),[§1](https://arxiv.org/html/2608.20771#S1.p4.1),[§2\.2](https://arxiv.org/html/2608.20771#S2.SS2.p1.1),[§3\.4](https://arxiv.org/html/2608.20771#S3.SS4.p3.1)\.
- Guoet al\.\(2025\)D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi, X\. Zhang, X\. Yu, Y\. Wu, Z\. F\. Wu, Z\. Gou, Z\. Shao, Z\. Li, Z\. Gao, A\. Liu, B\. Xue, B\. Wang, B\. Wu, B\. Feng, C\. Lu, C\. Zhao, C\. Deng, C\. Ruan, D\. Dai, D\. Chen, D\. Ji, E\. Li, F\. Lin, F\. Dai, F\. Luo, G\. Hao, G\. Chen, G\. Li, H\. Zhang, H\. Xu, H\. Ding, H\. Gao, H\. Qu, H\. Li, J\. Guo, J\. Li, J\. Chen, J\. Yuan, J\. Tu, J\. Qiu, J\. Li, J\. L\. Cai, J\. Ni, J\. Liang, J\. Chen, K\. Dong, K\. Hu, K\. You, K\. Gao, K\. Guan, K\. Huang, K\. Yu, L\. Wang, L\. Zhang, L\. Zhao, L\. Wang, L\. Zhang, L\. Xu, L\. Xia, M\. Zhang, M\. Zhang, M\. Tang, M\. Zhou, M\. Li, M\. Wang, M\. Li, N\. Tian, P\. Huang, P\. Zhang, Q\. Wang, Q\. Chen, Q\. Du, R\. Ge, R\. Zhang, R\. Pan, R\. Wang, R\. J\. Chen, R\. L\. Jin, R\. Chen, S\. Lu, S\. Zhou, S\. Chen, S\. Ye, S\. Wang, S\. Yu, S\. Zhou, S\. Pan, S\. S\. Li, S\. Zhou, S\. Wu, T\. Yun, T\. Pei, T\. Sun, T\. Wang, W\. Zeng, W\. Liu, W\. Liang, W\. Gao, W\. Yu, W\. Zhang, W\. L\. Xiao, W\. An, X\. Liu, X\. Wang, X\. Chen, X\. Nie, X\. Cheng, X\. Liu, X\. Xie, X\. Liu, X\. Yang, X\. Li, X\. Su, X\. Lin, X\. Q\. Li, X\. Jin, X\. Shen, X\. Chen, X\. Sun, X\. Wang, X\. Song, X\. Zhou, X\. Wang, X\. Shan, Y\. K\. Li, Y\. Q\. Wang, Y\. X\. Wei, Y\. Zhang, Y\. Xu, Y\. Li, Y\. Zhao, Y\. Sun, Y\. Wang, Y\. Yu, Y\. Zhang, Y\. Shi, Y\. Xiong, Y\. He, Y\. Piao, Y\. Wang, Y\. Tan, Y\. Ma, Y\. Liu, Y\. Guo, Y\. Ou, Y\. Wang, Y\. Gong, Y\. Zou, Y\. He, Y\. Xiong, Y\. Luo, Y\. You, Y\. Liu, Y\. Zhou, Y\. X\. Zhu, Y\. Huang, Y\. Li, Y\. Zheng, Y\. Zhu, Y\. Ma, Y\. Tang, Y\. Zha, Y\. Yan, Z\. Z\. Ren, Z\. Ren, Z\. Sha, Z\. Fu, Z\. Xu, Z\. Xie, Z\. Zhang, Z\. Hao, Z\. Ma, Z\. Yan, Z\. Wu, Z\. Gu, Z\. Zhu, Z\. Liu, Z\. Li, Z\. Xie, Z\. Song, Z\. Pan, Z\. Huang, Z\. Xu, Z\. Zhang, and Z\. ZhangDeepSeek\-r1 incentivizes reasoning in llms through reinforcement learning\.Nature645\(8081\),pp\. 633–638\.External Links:ISSN 1476\-4687,[Link](http://dx.doi.org/10.1038/s41586-025-09422-z),[Document](https://dx.doi.org/10.1038/s41586-025-09422-z)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px2.p1.1)\.
- Guoet al\.\(2026\)Y\. Guo, W\. Yang, S\. Yang, Z\. Liu, C\. Chen, Y\. Wei, Y\. Hu, Y\. Huang, G\. Hao, D\. Yuan, J\. Wang, X\. Chen, H\. Yu, L\. Lei, and P\. DiOpAgent: operator agent for web navigation\.External Links:2602\.13559,[Link](https://arxiv.org/abs/2602.13559)Cited by:[§5\.2](https://arxiv.org/html/2608.20771#S5.SS2.p1.1)\.
- Heet al\.\(2026\)B\. He, M\. Hu, Z\. Xu, H\. Wang, L\. Zong, Y\. Chen, C\. Ma, X\. Liu, P\. Zhou, and I\. KingSearch\-r2: enhancing search\-integrated reasoning via actor\-refiner collaboration\.External Links:2602\.03647,[Link](https://arxiv.org/abs/2602.03647)Cited by:[§1](https://arxiv.org/html/2608.20771#S1.p1.1),[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2608.20771#S5.SS1.p1.1),[§5\.2](https://arxiv.org/html/2608.20771#S5.SS2.p1.1)\.
- Hoet al\.\(2020\)X\. Ho, A\. D\. Nguyen, S\. Sugawara, and A\. AizawaConstructing a multi\-hop qa dataset for comprehensive evaluation of reasoning steps\.External Links:2011\.01060,[Link](https://arxiv.org/abs/2011.01060)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px1.p1.1)\.
- Jeonget al\.\(2024\)S\. Jeong, J\. Baek, S\. Cho, S\. J\. Hwang, and J\. C\. ParkAdaptive\-rag: learning to adapt retrieval\-augmented large language models through question complexity\.External Links:2403\.14403,[Link](https://arxiv.org/abs/2403.14403)Cited by:[§5\.1](https://arxiv.org/html/2608.20771#S5.SS1.p1.1)\.
- Jinet al\.\(2025\)B\. Jin, H\. Zeng, Z\. Yue, J\. Yoon, S\. Arik, D\. Wang, H\. Zamani, and J\. HanSearch\-r1: training llms to reason and leverage search engines with reinforcement learning\.External Links:2503\.09516,[Link](https://arxiv.org/abs/2503.09516)Cited by:[Appendix D](https://arxiv.org/html/2608.20771#A4.p1.1),[§1](https://arxiv.org/html/2608.20771#S1.p1.1),[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2608.20771#S5.SS1.p1.1),[§5\.2](https://arxiv.org/html/2608.20771#S5.SS2.p1.1)\.
- Joshiet al\.\(2017\)M\. Joshi, E\. Choi, D\. S\. Weld, and L\. ZettlemoyerTriviaQA: a large scale distantly supervised challenge dataset for reading comprehension\.External Links:1705\.03551,[Link](https://arxiv.org/abs/1705.03551)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px1.p1.1)\.
- Karpukhinet al\.\(2020\)V\. Karpukhin, B\. Oğuz, S\. Min, P\. Lewis, L\. Wu, S\. Edunov, D\. Chen, and W\. YihDense passage retrieval for open\-domain question answering\.External Links:2004\.04906,[Link](https://arxiv.org/abs/2004.04906)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px3.p1.1)\.
- Kumaret al\.\(2023\)B\. Kumar, C\. Lu, G\. Gupta, A\. Palepu, D\. Bellamy, R\. Raskar, and A\. BeamConformal prediction with large language models for multi\-choice question answering\.External Links:2305\.18404,[Link](https://arxiv.org/abs/2305.18404)Cited by:[§1](https://arxiv.org/html/2608.20771#S1.p3.1),[§5\.3](https://arxiv.org/html/2608.20771#S5.SS3.p1.1)\.
- Kwiatkowskiet al\.\(2019\)T\. Kwiatkowski, J\. Palomaki, O\. Redfield, M\. Collins, A\. Parikh, C\. Alberti, D\. Epstein, I\. Polosukhin, J\. Devlin, K\. Lee, K\. Toutanova, L\. Jones, M\. Kelcey, M\. Chang, A\. M\. Dai, J\. Uszkoreit, Q\. Le, and S\. PetrovNatural questions: a benchmark for question answering research\.Transactions of the Association for Computational Linguistics7,pp\. 452–466\.External Links:[Link](https://aclanthology.org/Q19-1026/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00276)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px1.p1.1)\.
- Leiet al\.\(2017\)J\. Lei, M\. G’Sell, A\. Rinaldo, R\. J\. Tibshirani, and L\. WassermanDistribution\-free predictive inference for regression\.External Links:1604\.04173,[Link](https://arxiv.org/abs/1604.04173)Cited by:[§B\.1](https://arxiv.org/html/2608.20771#A2.SS1.p10.2.1)\.
- Lenget al\.\(2025\)J\. Leng, C\. Huang, B\. Zhu, and J\. HuangTaming overconfidence in llms: reward calibration in rlhf\.External Links:2410\.09724,[Link](https://arxiv.org/abs/2410.09724)Cited by:[§1](https://arxiv.org/html/2608.20771#S1.p2.1),[§5\.2](https://arxiv.org/html/2608.20771#S5.SS2.p1.1)\.
- Lewiset al\.\(2021\)P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. KielaRetrieval\-augmented generation for knowledge\-intensive nlp tasks\.External Links:2005\.11401,[Link](https://arxiv.org/abs/2005.11401)Cited by:[§1](https://arxiv.org/html/2608.20771#S1.p1.1),[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2608.20771#S5.SS1.p1.1)\.
- Liet al\.\(2025\)X\. Li, G\. Dong, J\. Jin, Y\. Zhang, Y\. Zhou, Y\. Zhu, P\. Zhang, and Z\. DouSearch\-o1: agentic search\-enhanced large reasoning models\.External Links:2501\.05366,[Link](https://arxiv.org/abs/2501.05366)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2608.20771#S5.SS1.p1.1)\.
- Lianget al\.\(2026\)T\. Liang, Y\. Chen, S\. Ji, Y\. Chen, Z\. Jia, J\. Lu, F\. Zhuo, X\. Pu, Y\. Li, and Z\. ZhaoVoxMind: an end\-to\-end agentic spoken dialogue system\.External Links:2604\.15710,[Link](https://arxiv.org/abs/2604.15710)Cited by:[§1](https://arxiv.org/html/2608.20771#S1.p1.1)\.
- Liuet al\.\(2023\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the middle: how language models use long contexts\.External Links:2307\.03172,[Link](https://arxiv.org/abs/2307.03172)Cited by:[§1](https://arxiv.org/html/2608.20771#S1.p2.1)\.
- Luoet al\.\(2025\)J\. Luo, W\. Zhang, Y\. Yuan, Y\. Zhao, J\. Yang, Y\. Gu, B\. Wu, B\. Chen, Z\. Qiao, Q\. Long, R\. Tu, X\. Luo, W\. Ju, Z\. Xiao, Y\. Wang, M\. Xiao, C\. Liu, J\. Yuan, S\. Zhang, Y\. Jin, F\. Zhang, X\. Wu, H\. Zhao, D\. Tao, P\. S\. Yu, and M\. ZhangLarge language model agent: a survey on methodology, applications and challenges\.External Links:2503\.21460,[Link](https://arxiv.org/abs/2503.21460)Cited by:[§5\.2](https://arxiv.org/html/2608.20771#S5.SS2.p1.1)\.
- Mallenet al\.\(2023\)A\. Mallen, A\. Asai, V\. Zhong, R\. Das, D\. Khashabi, and H\. HajishirziWhen not to trust language models: investigating effectiveness of parametric and non\-parametric memories\.External Links:2212\.10511,[Link](https://arxiv.org/abs/2212.10511)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px1.p1.1)\.
- Presset al\.\(2023\)O\. Press, M\. Zhang, S\. Min, L\. Schmidt, N\. A\. Smith, and M\. LewisMeasuring and narrowing the compositionality gap in language models\.External Links:2210\.03350,[Link](https://arxiv.org/abs/2210.03350)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px1.p1.1)\.
- Quachet al\.\(2024\)V\. Quach, A\. Fisch, T\. Schuster, A\. Yala, J\. H\. Sohn, T\. S\. Jaakkola, and R\. BarzilayConformal language modeling\.External Links:2306\.10193,[Link](https://arxiv.org/abs/2306.10193)Cited by:[§3\.4](https://arxiv.org/html/2608.20771#S3.SS4.p2.1),[§5\.3](https://arxiv.org/html/2608.20771#S5.SS3.p1.1)\.
- Qwenet al\.\(2025\)Qwen, :, A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. QiuQwen2\.5 technical report\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px3.p1.1)\.
- Romanoet al\.\(2020\)Y\. Romano, M\. Sesia, and E\. J\. CandèsClassification with valid and adaptive coverage\.External Links:2006\.02544,[Link](https://arxiv.org/abs/2006.02544)Cited by:[§B\.2](https://arxiv.org/html/2608.20771#A2.SS2.p1.1),[§B\.2](https://arxiv.org/html/2608.20771#A2.SS2.p9.1),[§1](https://arxiv.org/html/2608.20771#S1.p4.1),[§2\.1](https://arxiv.org/html/2608.20771#S2.SS1.p1.1),[§5\.3](https://arxiv.org/html/2608.20771#S5.SS3.p1.1)\.
- Schicket al\.\(2023\)T\. Schick, J\. Dwivedi\-Yu, R\. Dessì, R\. Raileanu, M\. Lomeli, L\. Zettlemoyer, N\. Cancedda, and T\. ScialomToolformer: language models can teach themselves to use tools\.External Links:2302\.04761,[Link](https://arxiv.org/abs/2302.04761)Cited by:[§1](https://arxiv.org/html/2608.20771#S1.p1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.External Links:2402\.03300,[Link](https://arxiv.org/abs/2402.03300)Cited by:[§1](https://arxiv.org/html/2608.20771#S1.p4.1),[§3\.4](https://arxiv.org/html/2608.20771#S3.SS4.p4.1)\.
- Singhet al\.\(2026\)A\. Singh, A\. Ehtesham, S\. Kumar, T\. T\. Khoei, and A\. V\. VasilakosAgentic retrieval\-augmented generation: a survey on agentic rag\.External Links:2501\.09136,[Link](https://arxiv.org/abs/2501.09136)Cited by:[§5\.1](https://arxiv.org/html/2608.20771#S5.SS1.p1.1),[§5\.2](https://arxiv.org/html/2608.20771#S5.SS2.p1.1)\.
- Suet al\.\(2025\)J\. Su, F\. Lin, Z\. Feng, H\. Zheng, T\. Wang, Z\. Xiao, X\. Zhao, Z\. Liu, L\. Cheng, and H\. WangCP\-router: an uncertainty\-aware router between llm and lrm\.External Links:2505\.19970,[Link](https://arxiv.org/abs/2505.19970)Cited by:[§1](https://arxiv.org/html/2608.20771#S1.p3.1),[§5\.3](https://arxiv.org/html/2608.20771#S5.SS3.p1.1)\.
- Taoet al\.\(2025\)W\. Tao, X\. Xing, Z\. Li, and X\. XuSAKI\-RAG: mitigating context fragmentation in long\-document RAG via sentence\-level attention knowledge integration\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 1195–1213\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.63/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.63),ISBN 979\-8\-89176\-332\-6Cited by:[§5\.1](https://arxiv.org/html/2608.20771#S5.SS1.p1.1)\.
- Trivediet al\.\(2022\)H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. SabharwalMuSiQue: multihop questions via single\-hop question composition\.External Links:2108\.00573,[Link](https://arxiv.org/abs/2108.00573)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px1.p1.1)\.
- Trivediet al\.\(2023\)H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. SabharwalInterleaving retrieval with chain\-of\-thought reasoning for knowledge\-intensive multi\-step questions\.External Links:2212\.10509,[Link](https://arxiv.org/abs/2212.10509)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px2.p1.1)\.
- Vovket al\.\(2005\)V\. Vovk, A\. Gammerman, and G\. ShaferAlgorithmic learning in a random world\.Springer\-Verlag,Berlin, Heidelberg\.External Links:ISBN 0387001522Cited by:[§5\.3](https://arxiv.org/html/2608.20771#S5.SS3.p1.1)\.
- Wanget al\.\(2024\)L\. Wang, N\. Yang, X\. Huang, B\. Jiao, L\. Yang, D\. Jiang, R\. Majumder, and F\. WeiText embeddings by weakly\-supervised contrastive pre\-training\.External Links:2212\.03533,[Link](https://arxiv.org/abs/2212.03533)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px3.p1.1)\.
- Weiet al\.\(2023\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. Chi, Q\. Le, and D\. ZhouChain\-of\-thought prompting elicits reasoning in large language models\.External Links:2201\.11903,[Link](https://arxiv.org/abs/2201.11903)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px2.p1.1)\.
- Weiet al\.\(2025\)Y\. Wei, O\. Duchenne, J\. Copet, Q\. Carbonneaux, L\. Zhang, D\. Fried, G\. Synnaeve, R\. Singh, and S\. I\. WangSWE\-rl: advancing llm reasoning via reinforcement learning on open software evolution\.External Links:2502\.18449,[Link](https://arxiv.org/abs/2502.18449)Cited by:[§5\.2](https://arxiv.org/html/2608.20771#S5.SS2.p1.1)\.
- Wuet al\.\(2026\)Z\. Wu, S\. W\. Jeong, Y\. Liu, Y\. J\. Jung, and C\. DonnatFiltering with confidence: when data augmentation meets conformal prediction\.External Links:2509\.21479,[Link](https://arxiv.org/abs/2509.21479)Cited by:[§5\.3](https://arxiv.org/html/2608.20771#S5.SS3.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. QiuQwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px3.p1.1)\.
- Yanget al\.\(2018\)Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. W\. Cohen, R\. Salakhutdinov, and C\. D\. ManningHotpotQA: a dataset for diverse, explainable multi\-hop question answering\.External Links:1809\.09600,[Link](https://arxiv.org/abs/1809.09600)Cited by:[§4\.1](https://arxiv.org/html/2608.20771#S4.SS1.SSS0.Px1.p1.1)\.
- Yaoet al\.\(2023\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.External Links:2210\.03629,[Link](https://arxiv.org/abs/2210.03629)Cited by:[§1](https://arxiv.org/html/2608.20771#S1.p1.1)\.
- Yinet al\.\(2026\)C\. Yin, Z\. Sha, S\. Cui, C\. Meng, and Z\. LiThe reasoning trap: how enhancing llm reasoning amplifies tool hallucination\.External Links:2510\.22977,[Link](https://arxiv.org/abs/2510.22977)Cited by:[§1](https://arxiv.org/html/2608.20771#S1.p2.1)\.
- Zhanget al\.\(2026\)Z\. Zhang, Y\. Duan, Y\. Zhang, Y\. Xu, Z\. Wang, K\. Liang, Y\. Li, J\. Liang, D\. Xia, J\. Huang, J\. He, and Y\. WuOne tool is enough: reinforcement learning for repository\-level llm agents\.External Links:2512\.20957,[Link](https://arxiv.org/abs/2512.20957)Cited by:[§5\.2](https://arxiv.org/html/2608.20771#S5.SS2.p1.1)\.
- Zhaoet al\.\(2025\)S\. Zhao, T\. Yu, A\. Xu, J\. Singh, A\. Shukla, and R\. AkkirajuParallelSearch: train your llms to decompose query and search sub\-queries in parallel with reinforcement learning\.External Links:2508\.09303,[Link](https://arxiv.org/abs/2508.09303)Cited by:[§1](https://arxiv.org/html/2608.20771#S1.p1.1),[§5\.1](https://arxiv.org/html/2608.20771#S5.SS1.p1.1),[§5\.2](https://arxiv.org/html/2608.20771#S5.SS2.p1.1)\.

## Appendix APseudocode

We provide the pseudocode of CAS in Algorithm[1](https://arxiv.org/html/2608.20771#algorithm1)

Input:Dataset

𝒟\\mathcal\{D\}, Initial policy

πθ\\pi\_\{\\theta\}, Search engine

𝒮\\mathcal\{S\}
Parameters:Group size

GG, ACI step size

γ\\gamma, Target error rate

ρ\\rho, Initial calibration set size

NcalN\_\{\\text\{cal\}\}
Output:Optimized policy

πθ\\pi\_\{\\theta\}
1

Initialize ACI threshold

α1←ρ\\alpha\_\{1\}\\leftarrow\\rho;2

Initialize calibration set

𝒞\\mathcal\{C\}with

NcalN\_\{\\text\{cal\}\}scores from untrained policy;3

4

for*each RL training iterationt=1,2,…t=1,2,\\dots*do5

Sample a prompt

xt∼𝒟x\_\{t\}\\sim\\mathcal\{D\};6

7

for*trajectoryi=1i=1toGG*do8

while*trajectoryyiy\_\{i\}not terminated*do9

Generate tokens

yn​e​x​t∼πθ\(⋅∣xt,yi\)y\_\{next\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid x\_\{t\},y\_\{i\}\);10

if*search queryqqgenerated*then11

DAPS←APS​\(𝒮​\(q\)\)D\_\{\\text\{APS\}\}\\leftarrow\\text\{APS\}\(\\mathcal\{S\}\(q\)\)\(Sec\.[3\.2](https://arxiv.org/html/2608.20771#S3.SS2)\);12

yi←yi⊕DAPSy\_\{i\}\\leftarrow y\_\{i\}\\oplus D\_\{\\text\{APS\}\};13

end if14

end while15

end for16

17

for*trajectoryi=1i=1toGG*do18

Compute reward

Ri=racc\+rfmtR\_\{i\}=r\_\{\\text\{acc\}\}\+r\_\{\\text\{fmt\}\};19

Compute NLL score

s⁡\(xt,yi\)s\(x\_\{t\},y\_\{i\}\)for answer tokens \(Eq\. \([6](https://arxiv.org/html/2608.20771#S3.E6)\)\);20

end for21

Compute advantages

A1,…,AGA\_\{1\},\\dots,A\_\{G\}from rewards

\{Ri\}\\\{R\_\{i\}\\\};22

23

Compute quantile threshold

q^t←Q^t​\(1−αt\)\\hat\{q\}\_\{t\}\\leftarrow\\hat\{Q\}\_\{t\}\(1\-\\alpha\_\{t\}\)over

𝒞\\mathcal\{C\};24

25

for*trajectoryi=1i=1toGG*do26

Determine ACI confidence weight

ωi\\omega\_\{i\}using

q^t\\hat\{q\}\_\{t\}\(Eq\. \([9](https://arxiv.org/html/2608.20771#S3.E9)\)\);27

end for28

𝒞←𝒞∪\{s⁡\(xt,y1\),…,s⁡\(xt,yG\)\}\\mathcal\{C\}\\leftarrow\\mathcal\{C\}\\cup\\\{s\(x\_\{t\},y\_\{1\}\),\\dots,s\(x\_\{t\},y\_\{G\}\)\\\};29

Update

πθ\\pi\_\{\\theta\}by minimizing the ACI\-guided objective \(Eq\. \([8](https://arxiv.org/html/2608.20771#S3.E8)\)\);30

Update ACI threshold

αt\+1\\alpha\_\{t\+1\}\(Eq\. \([7](https://arxiv.org/html/2608.20771#S3.E7)\)\);31

end for32

Algorithm 1CAS
## Appendix BMathematical Proofs

In this section, we provide the formal mathematical proofs for the statistical guarantees of the CP methods used in CAS\. We begin with the fundamental marginal coverage guarantee of standard Split CP\.

### B\.1Marginal Coverage Guarantee of Conformal Prediction

Following the standard proof of validity for split\-conformal prediction\([3](https://arxiv.org/html/2608.20771#bib.bib9)\), we demonstrate that the prediction sets constructed via CP possess a strict, finite\-sample marginal coverage guarantee\.

Theorem 1\(Conformal calibration coverage guarantee\)\.Suppose the calibration data\(Xi,Yi\)i=1,…,n\(X\_\{i\},Y\_\{i\}\)\_\{i=1,\\dots,n\}and the test point\(Xtest,Ytest\)\(X\_\{\\text\{test\}\},Y\_\{\\text\{test\}\}\)are independent and identically distributed \(i\.i\.d\.\)\. Define the conformal quantileq^\\hat\{q\}as:

q^=inf\{q:\|\{i:s⁡\(Xi,Yi\)≤q\}\|n≥⌈\(n\+1\)​\(1−α\)⌉n\}\\hat\{q\}=\\inf\\left\\\{q:\\frac\{\|\\\{i:s\(X\_\{i\},Y\_\{i\}\)\\leq q\\\}\|\}\{n\}\\geq\\frac\{\\lceil\(n\+1\)\(1\-\\alpha\)\\rceil\}\{n\}\\right\\\}\(10\)and the resulting prediction sets as:

𝒞⁡\(X\)=\{y:s⁡\(X,y\)≤q^\}\\mathcal\{C\}\(X\)=\\\{y:s\(X,y\)\\leq\\hat\{q\}\\\}\(11\)Then, the marginal coverage satisfies:

ℙ⁡\(Ytest∈𝒞⁡\(Xtest\)\)≥1−α\\mathbb\{P\}\(Y\_\{\\text\{test\}\}\\in\\mathcal\{C\}\(X\_\{\\text\{test\}\}\)\)\\geq 1\-\\alpha\(12\)
Proof of Theorem 1\.Letsi=s⁡\(Xi,Yi\)s\_\{i\}=s\(X\_\{i\},Y\_\{i\}\)fori=1,…,ni=1,\\dots,nandstest=s⁡\(Xtest,Ytest\)s\_\{\\text\{test\}\}=s\(X\_\{\\text\{test\}\},Y\_\{\\text\{test\}\}\)\. To avoid handling ties, we consider the case where the non\-conformity scoressis\_\{i\}are distinct with probability 1\.

Without loss of generality, we assume the calibration scores are sorted such thats1<s2<⋯<sns\_\{1\}<s\_\{2\}<\\dots<s\_\{n\}\. In this case, the quantileq^\\hat\{q\}can be explicitly written as:

q^=s⌈\(n\+1\)​\(1−α\)⌉\\hat\{q\}=s\_\{\\lceil\(n\+1\)\(1\-\\alpha\)\\rceil\}\(13\)whenα≥1n\+1\\alpha\\geq\\frac\{1\}\{n\+1\}, andq^=∞\\hat\{q\}=\\inftyotherwise\.

Note that in the case whereq^=∞\\hat\{q\}=\\infty, the prediction set includes the entire label space, i\.e\.,𝒞⁡\(Xtest\)=𝒴\\mathcal\{C\}\(X\_\{\\text\{test\}\}\)=\\mathcal\{Y\}, so the coverage property is trivially satisfied\. Thus, we only need to handle the case whenα≥1n\+1\\alpha\\geq\\frac\{1\}\{n\+1\}\.

We proceed by noticing the strict equality of the two following events:

\{Ytest∈𝒞\(Xtest\)\}=\{stest≤q^\}\\\{Y\_\{\\text\{test\}\}\\in\\mathcal\{C\}\(X\_\{\\text\{test\}\}\)\\\}=\\\{s\_\{\\text\{test\}\}\\leq\\hat\{q\}\\\}\(14\)
Combining this with our definition of the sorted quantileq^\\hat\{q\}yields:

\{Ytest∈𝒞\(Xtest\)\}=\{stest≤s⌈\(n\+1\)​\(1−α\)⌉\}\\\{Y\_\{\\text\{test\}\}\\in\\mathcal\{C\}\(X\_\{\\text\{test\}\}\)\\\}=\\\{s\_\{\\text\{test\}\}\\leq s\_\{\\lceil\(n\+1\)\(1\-\\alpha\)\\rceil\}\\\}\(15\)
Now comes the crucial insight: by the exchangeability of the random variables\(X1,Y1\),…,\(Xtest,Ytest\)\(X\_\{1\},Y\_\{1\}\),\\dots,\(X\_\{\\text\{test\}\},Y\_\{\\text\{test\}\}\), their corresponding non\-conformity scoress1,…,sn,stests\_\{1\},\\dots,s\_\{n\},s\_\{\\text\{test\}\}are also exchangeable\. Because they are exchangeable,stests\_\{\\text\{test\}\}is equally likely to fall anywhere between the sorted calibration pointss1,…,sns\_\{1\},\\dots,s\_\{n\}\. Therefore, the probability thatstests\_\{\\text\{test\}\}is less than or equal to thekk\-th sorted score is exactly:

ℙ⁡\(stest≤sk\)=kn\+1\\mathbb\{P\}\(s\_\{\\text\{test\}\}\\leq s\_\{k\}\)=\\frac\{k\}\{n\+1\}\(16\)for any integerkk\. \(Note that here, the randomness is over all variabless1,…,sn,stests\_\{1\},\\dots,s\_\{n\},s\_\{\\text\{test\}\}\)\.

From this property, we substitutek=⌈\(n\+1\)​\(1−α\)⌉k=\\lceil\(n\+1\)\(1\-\\alpha\)\\rceilto conclude:

ℙ⁡\(stest≤s⌈\(n\+1\)​\(1−α\)⌉\)=⌈\(n\+1\)​\(1−α\)⌉n\+1≥1−α\\mathbb\{P\}\\left\(s\_\{\\text\{test\}\}\\leq s\_\{\\lceil\(n\+1\)\(1\-\\alpha\)\\rceil\}\\right\)=\\frac\{\\lceil\(n\+1\)\(1\-\\alpha\)\\rceil\}\{n\+1\}\\geq 1\-\\alpha\(17\)which implies the desired result:ℙ⁡\(Ytest∈𝒞⁡\(Xtest\)\)≥1−α\\mathbb\{P\}\(Y\_\{\\text\{test\}\}\\in\\mathcal\{C\}\(X\_\{\\text\{test\}\}\)\)\\geq 1\-\\alpha\.

Theorem 2\(Conformal calibration upper bound\)\.Additionally, if the scoress1,…,sns\_\{1\},\\dots,s\_\{n\}have a continuous joint distribution \(i\.e\., avoiding ties\), the coverage is tightly bounded from above:

ℙ⁡\(Ytest∈𝒞⁡\(Xtest\)\)≤1−α\+1n\+1\\mathbb\{P\}\(Y\_\{\\text\{test\}\}\\in\\mathcal\{C\}\(X\_\{\\text\{test\}\}\)\)\\leq 1\-\\alpha\+\\frac\{1\}\{n\+1\}\(18\)\(Proof deferred to Theorem 2\.2 of[22](https://arxiv.org/html/2608.20771#bib.bib47)\)\.

### B\.2Statistical Guarantees of Adaptive Prediction Sets

As established in the foundational literature, achieving exact finite\-sample conditional coverage is theoretically impossible without strong distributional assumptions\. However, the APS framework[33](https://arxiv.org/html/2608.20771#bib.bib17)effectively circumvents this limitation\. It provides a rigorous marginal coverage guarantee while sensibly approximating conditional coverage by adapting the prediction set size to the local uncertainty of the input\.

To construct the adaptive sets, APS introduces a generalized inverse quantile conformity score\. Given a base model’s probability estimateπ^\\hat\{\\pi\}and a uniform random variableU∼Uniform​\(0,1\)U\\sim\\text\{Uniform\}\(0,1\)for tie\-breaking, the conformity score functionEEis defined as:

E⁡\(x,y,u,π^\)=min⁡\{τ∈\[0,1\]:y∈𝒮⁡\(x,u,π^,τ\)\}E\(x,y,u;\\hat\{\\pi\}\)=\\min\\\{\\tau\\in\[0,1\]:y\\in\\mathcal\{S\}\(x,u;\\hat\{\\pi\},\\tau\)\\\}\(19\)where𝒮\\mathcal\{S\}is the generalized conditional quantile function that includes classes in descending order of their estimated probabilities until the cumulative mass reachesτ\\tau\.

Using this conformity score, APS achieves the following rigorous marginal guarantee:

Theorem 3\(Marginal coverage of APS\)\.If the calibration samples\(Xi,Yi\)i∈ℐ2\(X\_\{i\},Y\_\{i\}\)\_\{i\\in\\mathcal\{I\}\_\{2\}\}and the test sample\(Xtest,Ytest\)\(X\_\{\\text\{test\}\},Y\_\{\\text\{test\}\}\)are exchangeable, and the conformity scores are calculated using a model trained on a disjoint splitℐ1\\mathcal\{I\}\_\{1\}, the APS prediction set𝒞^APS\\hat\{\\mathcal\{C\}\}\_\{\\text\{APS\}\}satisfies:

ℙ⁡\(Ytest∈𝒞^APS​\(Xtest\)\)≥1−α\\mathbb\{P\}\\left\(Y\_\{\\text\{test\}\}\\in\\hat\{\\mathcal\{C\}\}\_\{\\text\{APS\}\}\(X\_\{\\text\{test\}\}\)\\right\)\\geq 1\-\\alpha\(20\)Furthermore, if the scoresEiE\_\{i\}are almost surely distinct, the coverage is bounded tightly from above by1−α\+1/\(\|ℐ2\|\+1\)1\-\\alpha\+1/\(\|\\mathcal\{I\}\_\{2\}\|\+1\)\.

Proof of Theorem 3\.LetEi=E⁡\(Xi,Yi,Ui,π^\)E\_\{i\}=E\(X\_\{i\},Y\_\{i\},U\_\{i\};\\hat\{\\pi\}\)denote the conformity score for theii\-th calibration sample inℐ2\\mathcal\{I\}\_\{2\}, andEtest=E⁡\(Xtest,Ytest,Utest,π^\)E\_\{\\text\{test\}\}=E\(X\_\{\\text\{test\}\},Y\_\{\\text\{test\}\},U\_\{\\text\{test\}\};\\hat\{\\pi\}\)for the test point\.

By the construction of the APS prediction set, a labelyyis included in𝒞^APS​\(Xtest\)\\hat\{\\mathcal\{C\}\}\_\{\\text\{APS\}\}\(X\_\{\\text\{test\}\}\)if and only if its requisite cumulative massτ\\tauis less than or equal to the calibrated thresholdQ^1−α\\hat\{Q\}\_\{1\-\\alpha\}\. Mathematically, we know that:

Ytest∈𝒞^APS​\(Xtest\)⇔Etest≤Q^1−α​\(\{Ei\}i∈ℐ2\)Y\_\{\\text\{test\}\}\\in\\hat\{\\mathcal\{C\}\}\_\{\\text\{APS\}\}\(X\_\{\\text\{test\}\}\)\\iff E\_\{\\text\{test\}\}\\leq\\hat\{Q\}\_\{1\-\\alpha\}\(\\\{E\_\{i\}\\\}\_\{i\\in\\mathcal\{I\}\_\{2\}\}\)\(21\)whereQ^1−α​\(\{Ei\}i∈ℐ2\)\\hat\{Q\}\_\{1\-\\alpha\}\(\\\{E\_\{i\}\\\}\_\{i\\in\\mathcal\{I\}\_\{2\}\}\)is defined as the⌈\(1−α\)​\(1\+\|ℐ2\|\)⌉\\lceil\(1\-\\alpha\)\(1\+\|\\mathcal\{I\}\_\{2\}\|\)\\rceil\-th smallest value in the calibration score set\{Ei\}i∈ℐ2\\\{E\_\{i\}\\\}\_\{i\\in\\mathcal\{I\}\_\{2\}\}\.

Because the data points\(X,Y\)\(X,Y\)are exchangeable and the uniform variablesUUare i\.i\.d\., all the evaluated conformity scoresEtestE\_\{\\text\{test\}\}and\{Ei\}i∈ℐ2\\\{E\_\{i\}\\\}\_\{i\\in\\mathcal\{I\}\_\{2\}\}are completely exchangeable\. Under the property of exchangeability, the rank ofEtestE\_\{\\text\{test\}\}is uniformly distributed among the\|ℐ2\|\+1\|\\mathcal\{I\}\_\{2\}\|\+1scores\. Therefore, the probability of the event thatEtestE\_\{\\text\{test\}\}falls below the empirical\(1−α\)\(1\-\\alpha\)\-quantile is bounded from below by the nominal level:

ℙ⁡\(Etest≤Q^1−α​\(\{Ei\}i∈ℐ2\)\)≥1−α\\mathbb\{P\}\\left\(E\_\{\\text\{test\}\}\\leq\\hat\{Q\}\_\{1\-\\alpha\}\(\\\{E\_\{i\}\\\}\_\{i\\in\\mathcal\{I\}\_\{2\}\}\)\\right\)\\geq 1\-\\alpha\(22\)which immediately establishesℙ⁡\(Ytest∈𝒞^APS​\(Xtest\)\)≥1−α\\mathbb\{P\}\(Y\_\{\\text\{test\}\}\\in\\hat\{\\mathcal\{C\}\}\_\{\\text\{APS\}\}\(X\_\{\\text\{test\}\}\)\)\\geq 1\-\\alpha\.

Asymptotic Conditional Coverage\.While Theorem 3 guarantees marginal coverage, the structural design of APS provides an asymptotic approximation of conditional coverage\. Consider an Oracle model with perfect knowledge of the true conditional distributionπy​\(x\)=ℙ⁡\(Y=y\|X=x\)\\pi\_\{y\}\(x\)=\\mathbb\{P\}\(Y=y\|X=x\)\. The Oracle’s prediction set𝒞αoracle​\(x\)\\mathcal\{C\}\_\{\\alpha\}^\{\\text\{oracle\}\}\(x\)naturally attains exact conditional coverage\.

According to[33](https://arxiv.org/html/2608.20771#bib.bib17), as the sample size increases and if the base predictive model is consistent \(i\.e\.,π^y​\(x\)≈πy​\(x\)\\hat\{\\pi\}\_\{y\}\(x\)\\approx\\pi\_\{y\}\(x\)\), the constructed sets𝒮⁡\(X,U,π^,τ\)\\mathcal\{S\}\(X,U;\\hat\{\\pi\},\\tau\)will converge to contain the true labels for exactly a fractionτ\\tauof the points\. In this limit, the thresholdQ^1−α≈1−α\\hat\{Q\}\_\{1\-\\alpha\}\\approx 1\-\\alpha, and the decision rule approaches:

𝒞^APS​\(Xtest\)≈\{y∈𝒴:E⁡\(Xtest,y,Utest,π\)≤1−α\}\\hat\{\\mathcal\{C\}\}\_\{\\text\{APS\}\}\(X\_\{\\text\{test\}\}\)\\approx\\\{y\\in\\mathcal\{Y\}:E\(X\_\{\\text\{test\}\},y,U\_\{\\text\{test\}\};\\pi\)\\leq 1\-\\alpha\\\}\(23\)which mathematically equates to the exact output of the Oracle procedure, thereby closely approximating optimal conditional coverage in complex data scenarios\.

### B\.3Statistical Guarantees of Adaptive Conformal Inference

Standard CP fundamentally relies on the exchangeability of the data\. In online settings, the policy continuously evolves, leading to severe distribution shifts that violate the i\.i\.d\. assumption\. To maintain rigorous coverage, we employ ACI[11](https://arxiv.org/html/2608.20771#bib.bib13)\.

ACI guarantees the target coverage frequency over long\-time intervals irrespective of the true data\-generating process by dynamically adjusting the nominal error level\. Following[11](https://arxiv.org/html/2608.20771#bib.bib13), letρ∈\(0,1\)\\rho\\in\(0,1\)be the target miscoverage rate\. At each time steptt, the algorithm uses a parameterαt\\alpha\_\{t\}to construct the prediction set𝒞^t​\(αt\)\\hat\{\\mathcal\{C\}\}\_\{t\}\(\\alpha\_\{t\}\), and records the miscoverage event:

errt:=\{1,if​Yt∉𝒞^t​\(αt\)0,otherwise\\text\{err\}\_\{t\}:=\\begin\{cases\}1,&\\text\{if \}Y\_\{t\}\\notin\\hat\{\\mathcal\{C\}\}\_\{t\}\(\\alpha\_\{t\}\)\\\\ 0,&\\text\{otherwise\}\\end\{cases\}\(24\)The parameterαt\\alpha\_\{t\}is recursively updated using a step sizeγ\>0\\gamma\>0:

αt\+1:=αt\+γ⁡\(ρ−errt\)\\alpha\_\{t\+1\}:=\\alpha\_\{t\}\+\\gamma\(\\rho\-\\text\{err\}\_\{t\}\)\(25\)
To establish the distribution\-free guarantee, we assume that with probability one,α1∈\[0,1\]\\alpha\_\{1\}\\in\[0,1\]and the quantile functionQ^t​\(x\)\\hat\{Q\}\_\{t\}\(x\)is non\-decreasing withQ^t​\(x\)=−∞\\hat\{Q\}\_\{t\}\(x\)=\-\\inftyforx<0x<0andQ^t​\(x\)=∞\\hat\{Q\}\_\{t\}\(x\)=\\inftyforx\>1x\>1\.

Lemma 4\(Boundedness ofαt\\alpha\_\{t\}, Lemma 4\.1 in[11](https://arxiv.org/html/2608.20771#bib.bib13)\)\.With probability one, we have that for allt∈ℕt\\in\\mathbb\{N\},αt∈\[−γ,1\+γ\]\\alpha\_\{t\}\\in\[\-\\gamma,1\+\\gamma\]\.

Proof of Lemma 4\.Assume by contradiction that with positive probability, the sequence\{αt\}t∈ℕ\\\{\\alpha\_\{t\}\\\}\_\{t\\in\\mathbb\{N\}\}is such thatinftαt<−γ\\inf\_\{t\}\\alpha\_\{t\}<\-\\gamma\(the case forsuptαt\>1\+γ\\sup\_\{t\}\\alpha\_\{t\}\>1\+\\gammais symmetric\)\. Notice that the maximum change in one step is bounded:supt\|αt\+1−αt\|=suptγ​\|ρ−errt\|<γ\\sup\_\{t\}\|\\alpha\_\{t\+1\}\-\\alpha\_\{t\}\|=\\sup\_\{t\}\\gamma\|\\rho\-\\text\{err\}\_\{t\}\|<\\gamma\. Thus, with positive probability, we may find a specific time stept∈ℕt\\in\\mathbb\{N\}such thatαt<0\\alpha\_\{t\}<0andαt\+1<αt\\alpha\_\{t\+1\}<\\alpha\_\{t\}\.

However, by the boundary definition of the quantile function:

αt<0⟹Q^t​\(1−αt\)=∞⟹errt=0\\alpha\_\{t\}<0\\implies\\hat\{Q\}\_\{t\}\(1\-\\alpha\_\{t\}\)=\\infty\\implies\\text\{err\}\_\{t\}=0\(26\)Substitutingerrt=0\\text\{err\}\_\{t\}=0into the update rule gives:

αt\+1=αt\+γ⁡\(α−0\)≥αt\\alpha\_\{t\+1\}=\\alpha\_\{t\}\+\\gamma\(\\alpha\-0\)\\geq\\alpha\_\{t\}\(27\)This contradicts the assumption thatαt\+1<αt\\alpha\_\{t\+1\}<\\alpha\_\{t\}\. Thus,ℙ⁡\(∃t​such that​αt\+1<αt<0\)=0\\mathbb\{P\}\(\\exists t\\text\{ such that \}\\alpha\_\{t\+1\}<\\alpha\_\{t\}<0\)=0, establishing the lower bound\.

Theorem 5\(Distribution\-free asymptotic coverage, Proposition 4\.1 in[11](https://arxiv.org/html/2608.20771#bib.bib13)\)\.With probability one, for all horizon lengthsT∈ℕT\\in\\mathbb\{N\}, the empirical miscoverage rate satisfies:

\|1T​∑t=1Terrt−ρ\|≤max⁡\{α1,1−α1\}\+γT​γ\\left\|\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\text\{err\}\_\{t\}\-\\rho\\right\|\\leq\\frac\{\\max\\\{\\alpha\_\{1\},1\-\\alpha\_\{1\}\\\}\+\\gamma\}\{T\\gamma\}\(28\)In particular, asT→∞T\\to\\infty, the average miscoverage converges almost surely to the target rateα\\alpha:

limT→∞1T​∑t=1Terrt​=a\.s\.​ρ\\lim\_\{T\\to\\infty\}\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\text\{err\}\_\{t\}\\overset\{a\.s\.\}\{=\}\\rho\(29\)
Proof of Theorem 5\.By recursively expanding the update ruleαt\+1=αt\+γ⁡\(α−errt\)\\alpha\_\{t\+1\}=\\alpha\_\{t\}\+\\gamma\(\\alpha\-\\text\{err\}\_\{t\}\)fromt=1t=1toTT, we obtain the telescoping sum:

αT\+1=α1\+∑t=1Tγ⁡\(ρ−errt\)\\alpha\_\{T\+1\}=\\alpha\_\{1\}\+\\sum\_\{t=1\}^\{T\}\\gamma\(\\rho\-\\text\{err\}\_\{t\}\)\(30\)Rearranging the terms to isolate the empirical average oferrt\\text\{err\}\_\{t\}, we get:

1T​∑t=1T\(errt−ρ\)=α1−αT\+1T​γ\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\(\\text\{err\}\_\{t\}\-\\rho\)=\\frac\{\\alpha\_\{1\}\-\\alpha\_\{T\+1\}\}\{T\\gamma\}\(31\)Taking the absolute value on both sides yields:

\|1T​∑t=1Terrt−ρ\|=\|α1−αT\+1\|T​γ\\left\|\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\text\{err\}\_\{t\}\-\\rho\\right\|=\\frac\{\|\\alpha\_\{1\}\-\\alpha\_\{T\+1\}\|\}\{T\\gamma\}\(32\)From Lemma 4, we know thatαT\+1∈\[−γ,1\+γ\]\\alpha\_\{T\+1\}\\in\[\-\\gamma,1\+\\gamma\]\. Given that the initializationα1∈\[0,1\]\\alpha\_\{1\}\\in\[0,1\], the maximum possible distance betweenα1\\alpha\_\{1\}andαT\+1\\alpha\_\{T\+1\}is bounded by:

\|α1−αT\+1\|≤max⁡\{α1−\(−γ\),1\+γ−α1\}\|\\alpha\_\{1\}\-\\alpha\_\{T\+1\}\|\\leq\\max\\\{\\alpha\_\{1\}\-\(\-\\gamma\),1\+\\gamma\-\\alpha\_\{1\}\\\}\(33\)which simplifies tomax⁡\{α1,1−α1\}\+γ\\max\\\{\\alpha\_\{1\},1\-\\alpha\_\{1\}\\\}\+\\gamma\. Substituting this upper bound into the absolute difference completes the proof for Equation[28](https://arxiv.org/html/2608.20771#A2.E28)\.

Taking the limit asT→∞T\\to\\infty, the right\-hand side of Equation[28](https://arxiv.org/html/2608.20771#A2.E28)diminishes to zero \(sinceγ\\gammais a fixed positive constant\), proving that ACI flawlessly achieves the exact marginal coverage frequency over time, without making any assumptions on the nature of the data distribution shift\.

## Appendix CSupplementary Implementation Details

#### Environment

Our framework operates on a dual\-service architecture developed based on the VeRL distributed reinforcement learning framework\. The Training Service executes GRPO using Python 3\.12, PyTorch 2\.8\.0 \(CUDA 12\.9\), and is distributed across 4 GPUs via Ray \(v2\.49\.2\)\. To accelerate asynchronous multi\-turn rollouts, it leverages sglang \(v0\.5\.3rc0\) equipped with the flashinfer backend and flash\-attn \(v2\.8\.3\)\. The Retrieval Service operates independently as a FastAPI\-based REST endpoint using Python 3\.10 and PyTorch 2\.4\.0 \(CUDA 12\.1\)\. It utilizes faiss\-gpu \(v1\.8\.0\) and the e5\-base\-v2 embedding model, performing high\-throughput dense retrieval via mean pooling on 256\-token inputs with FP16 precision\. The retrieval backend is configured to handle a peak rate of 120 queries per second \(QPS\) with a 30\-second timeout\.

#### Configurations

This encompasses our data processing, optimization, and CP settings\. Data & Rollout: Models are trained on a unified search\-integrated reasoning dataset in Parquet format\. We set the maximum prompt, response, and context lengths to 4096, 3000, and 15,000 tokens, respectively, filtering out prompts that exceed the limit\. During the GRPO step, we sampleG=5G=5trajectories per prompt with a maximum of 4 assistant turns\. Optimization: The Actor is optimized with a learning rate of1×10−61\\times 10^\{\-6\}and a warmup ratio of 0\.285 \(100 steps\), while the Critic uses1×10−51\\times 10^\{\-5\}\. Training employs a global batch size of 512, a low\-variance KL penalty coefficient of 0\.001, and Fully Sharded Data Parallel \(FSDP\) with tensor model parallelism set to 1\. Reward Design: The rule\-based reward comprises an EM accuracy score \(weight 1\.0\) and format rewards \(0\.2 for structural integrity, 0\.1 for the final answer boundary\)\. CAS: On the retrieval side, APS are applied with a significance levelαA​P​S=0\.20\\alpha\_\{APS\}=0\.20and a temperature of 0\.01, dynamically restricting the retrieved subset to between 1 and 5 documents\. On the training side, ACI is initialized with a target error rateρ=0\.25\\rho=0\.25and an update step sizeγ=0\.005\\gamma=0\.005\. We apply a discount factorη=0\.5\\eta=0\.5to penalize low\-confidence trajectories \(si\>q^ts\_\{i\}\>\\hat\{q\}\_\{t\}\), while empirical error tracking utilizes an Exponential Moving Average \(EMA\) ratio of 0\.05 to maintain quantile stability\.

#### Hardware

All experiments were conducted on a single server node\. The server is configured with dual\-socket AMD EPYC 9454 48\-Core processors, providing a total of 96 physical cores and 192 threads, organized into two NUMA nodes\. The server is equipped with four NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs and 755 GiB of system memory\. Storage infrastructure includes a 446\.6 GB drive for the OS and environment, alongside a 14\.6 TB enterprise\-grade drive for high\-throughput data caching\. The software environment is built on Ubuntu 24\.04\.3 LTS\.

## Appendix DPrompts

In this section, we present the detailed prompt templates utilized across different stages of CAS\. The configuration of the reasoning template in Table[5](https://arxiv.org/html/2608.20771#A5.T5)is adapted from Search\-R1[17](https://arxiv.org/html/2608.20771#bib.bib4)to maintain consistency in agentic behavior\. Additionally, the query decomposition prompt in Table[6](https://arxiv.org/html/2608.20771#A5.T6)and the retrieval relevance judge prompt in Table[7](https://arxiv.org/html/2608.20771#A5.T7)are specifically employed to construct the calibration set for the retrieval\-side APS\.

## Appendix EAdditional Experimental Analysis on Qwen3\-8B

In Table[1](https://arxiv.org/html/2608.20771#S4.T1), it is observed that our method’s performance on Qwen3\-8B consistently outperforms that on Qwen2\.5\-3B\-Instruct\. This improvement is primarily attributed to the inherent model capacity of Qwen3\-8B, which exhibits a significant advantage over the 3B\-Instruct variant, as illustrated in Figure[3\(a\)](https://arxiv.org/html/2608.20771#A5.F3.sf1)\. Regarding the search behavior shown in Figure[3\(b\)](https://arxiv.org/html/2608.20771#A5.F3.sf2), we note that the average search calls for Qwen3\-8B remain lower than those of Qwen2\.5\-3B\-Instruct during approximately the first 80 training steps\. This phenomenon occurs because Qwen2\.5\-3B\-Instruct, as a smaller model, tends to exhibit erratic and indiscriminate tool invocation during the early stages of training\. In contrast, the larger parameter scale of Qwen3\-8B ensures more efficient search calls from the beginning\. This efficiency is further evidenced by comparing Figure[3\(a\)](https://arxiv.org/html/2608.20771#A5.F3.sf1)and Figure[3\(b\)](https://arxiv.org/html/2608.20771#A5.F3.sf2), where Qwen3\-8B achieves substantially higher EM scores despite a noticeably lower frequency of search invocations\. Furthermore, as depicted in Figure[3\(a\)](https://arxiv.org/html/2608.20771#A5.F3.sf1), although the number of search calls for Qwen3\-8B increases slightly relative to Qwen2\.5\-3B\-Instruct after convergence, it remains significantly more efficient than baseline methods lacking ACI constraints\. This demonstrates that the ACI mechanism effectively modulates low\-confidence trajectories even when applied to the 8B model\.

For the experiments involving Qwen3\-8B, the thinking mode is disabled by default, as enabling this feature leads to a drastic reduction in training effectiveness, as shown in Figure[3\(c\)](https://arxiv.org/html/2608.20771#A5.F3.sf3)\. The underlying cause is revealed in Figure[3\(d\)](https://arxiv.org/html/2608.20771#A5.F3.sf4): after enabling the thinking mode, the model initially tends towards aggressive search calls due to the interleaving of internal reasoning with our prescribed reasoning grammar\. However, the model rapidly discovers that many single\-hop problems can be resolved solely through internal reasoning\. Consequently, it gradually ceases to invoke the search tool, leading to a complete cessation of active information gathering and rendering the training process ineffective for the intended search\-integrated tasks\.

Table 5:Template for CAS reasoning process\.Table 6:Prompt for decomposing multi\-hop queries into single\-hop sub\-queries\.Table 7:Judge prompt for locating the most relevant documents\.\(a\)EM score during training\.\(b\)Average search calls\.\(c\)EM \(Thinking mode\)\.\(d\)Calls \(Thinking mode\)\.
Figure 3:Visualization of training metrics and the impact of the thinking mode\. Panels \(a\) and \(b\) illustrate the progression of accuracy and tool invocation frequency during training\. Panels \(c\) and \(d\) demonstrate how the thinking mode influences the final EM scores and the efficiency of the search process\.
## Appendix FDetailed Results for Ablation and Sensitivity Analysis

Corresponding to Table[3](https://arxiv.org/html/2608.20771#S4.T3)in the main text, we provide the complete results across all individual datasets in Table[8](https://arxiv.org/html/2608.20771#A6.T8)\.

Table 8:Detailed ablation study and sensitivity analysis on Qwen2\.5\-3B\-Instruct\. The table presents component\-wise ablations and sensitivity analyses forρ\\rhoandαAPS\\alpha\_\{\\text\{APS\}\}\.
## Appendix GCase Study

In this section, we present representative qualitative cases to illustrate the core behavioral patterns of CAS\.

Table[10](https://arxiv.org/html/2608.20771#A7.T10)illustrates a straightforward single\-hop scenario\. The model directly addresses the factual query by formulating a precise search action\. Because the retrieved documents are clean and highly relevant, the agent swiftly concludes its reasoning and extracts the correct answer without unnecessary actions\.

Table[9](https://arxiv.org/html/2608.20771#A7.T9)presents a more challenging single\-hop case characterized by high retrieval noise\. Although the search results contain highly distracting entities with similar names, the agent successfully evaluates the contextual relevance of each document, filters out the irrelevant distractors, and accurately grounds its final answer on the correct source\.

Table[11](https://arxiv.org/html/2608.20771#A7.T11)demonstrates the framework’s capability to handle multi\-hop queries through interleaved reasoning and search\. The agent dynamically decomposes the complex task in its initial<think\>block, retrieves the missing bridge entity in the first hop, and uses this intermediate information to construct a targeted query for the subsequent hop\. This iterative process highlights the effectiveness of allowing the model to flexibly transition between internal deliberation and external tool interaction\.

Question:What BBC television drama serial starring Jim McManus was based on Sarah Waters’ debut novel of the same name?Golden Answer:\[’Tipping the Velvet’\]<think\>The question asks for a BBC television drama serial based on Sarah Waters’ debut novel of the same name\. I recall that Sarah Waters’ debut novel is probably "Tipping the Velvet", but I need to confirm it, especially because Jim McManus may create ambiguity in search results\.</think\><search\>Jim McManus Tipping the Velvet</search\><information\>Doc 1 \(Title: "Jack McManus"\)Jack McManus Jack McManus \(born Thomas McManus, March 1862 – May 1905\), also known as Eat ’Em Up, was a noted New York City gangster around the turn of the 20th century\. Born in Boston, he was considered one of the premier boxers of the underworld, rivaled only by Monk Eastman\.Doc 2 \(Title: "Jimmy Velvet"\)toured in 1973\-1978 with the Jimmy Velvet Show band\. Band members included Steve Morgan \(guitar\), John Collins \(bassist\), Dennis Moog \(drums\), and Kathy Slinkard Velvet\. Velvet was championed by TV host Dick Clark, who had him on his "American Bandstand" show more than once\.Doc 3 \(Title: "Tipping the Velvet"\)Tipping the Velvet is a historical novel published as Sarah Waters’ debut novel in 1998\. Set in England during the 1890s, it tells a coming of age story about a young woman named Nan who falls in love with a male impersonator, follows her to London, and finds various ways to support herself as she journeys through the city\.Doc 4 \(Title: "James McManus"\)James "Jim" McManus \(born March 22, 1951\) is an American teacher, writer and poker player living in Kenilworth, Illinois\. McManus is best known as the author of the book "Positively Fifth"\.</information\><think\>The search results contain several noisy results about other people named McManus or Velvet, but Doc 3 directly confirms that "Tipping the Velvet" is Sarah Waters’ debut novel\. Since the question asks for the BBC drama serial based on that debut novel of the same name, the answer is "Tipping the Velvet"\.</think\><answer\>Tipping the Velvet</answer\>Table 9:Case Study: Complex Single\-Hop QA\.Question:Who won the Men’s Singles French Open Tennis Tournament in 2018?Golden Answer:\[’Rafael Nadal’\]<think\>I need to find out who won the Men’s Singles French Open Tennis Tournament in 2018\. I remember Rafael Nadal was dominant at the French Open around that time, but I should verify the exact 2018 result\.</think\><search\>2018 French Open Men’s Singles winner</search\><information\>Doc 1 \(Title: "2018 French Open – Men’s Singles"\)2018 French Open – Men’s Singles Rafael Nadal was the defending champion and successfully defended his title, defeating Dominic Thiem in the final, 6–4, 6–3, 6–2\. Nadal equaled Margaret Court’s all\-time record of 11 Grand Slam singles titles won at one major and became the first player to achieve that feat in the Open era\. He dropped only one set during the entire tournament \(losing the first set of his quarterfinal match against Diego Schwartzman\)\. Nadal also retained the ATP No\. 1 singles ranking by defending the title\.Doc 2 \(Title: "2018 French Open – Men’s Singles"\)72nd in the ATP rankings, became the first unseeded men’s singles semifinalist at the French Open since Gaël Monfils in 2008, the lowest\-ranked male singles player to reach the French Open semifinals since Andriy Medvedev in 1999, and the first Italian man to reach a Grand Slam singles semifinal since Corrado Barazzutti at the 1978 French Open\. 2018 French Open – Men’s Singles Rafael Nadal was the defending champion and successfully defended his title, defeating Dominic Thiem in the final, 6–4, 6–3, 6–2\.</information\><think\>The search results clearly state that Rafael Nadal won the 2018 French Open men’s singles title by defeating Dominic Thiem in the final\.</think\><answer\>Rafael Nadal</answer\>Table 10:Case Study: Simple Single\-Hop QA\.Question:Which hockey team does the younger brother of Sean James Pronger currently work for?Golden Answer:\[’the Florida Panthers’\]<think\>I need to identify Sean James Pronger’s younger brother first, then determine which hockey team that person currently works for\. This requires at least two steps: family relation lookup, then current team lookup\.</think\><search\>Sean James Pronger younger brother</search\><information\>Doc 1 \(Title: "Jordan Reaves"\)with Saskatchewan for 2018 training camp\. Jordan is the youngest son of former Winnipeg Blue Bomber running back Willard Reaves, who played with the team for five years and was a member of the 72nd Grey Cup winning team\. Jordan is also the younger brother of Ryan Reaves, a professional hockey player and right wing for the Vegas Golden Knights of the National Hockey League\.Doc 2 \(Title: "Chris Pronger"\)the 1990–91 season\. One of his defence partners in Stratford was future NHL player Greg de Vries\. In May 1991, Pronger indicated he was going to join his older brother Sean at Bowling Green State University \(NCAA\) instead of opting for the OHL\. Regardless of his pre\-draft indications, Pronger was selected in the 6th round by the Peterborough Petes in the OHL Priority Selection\.Doc 3 \(Title: "Luke Schenn"\)hits\. He believes he is at his best when he is playing a tough game, while staying responsible\. Growing up, Schenn patterned his play after fellow defencemen Chris Pronger and Rob Blake\. Schenn was born on November 2, 1989, in Saskatoon, Saskatchewan\. His younger brother Brayden was picked fifth overall in the 2009 NHL Entry Draft by the Los Angeles Kings\.Doc 4 \(Title: "Jordan Eberle"\)two sisters, Ashley and Whitney, and a younger brother, Dustin\. He attended high school at the Athol Murray College of Notre Dame in Wilcox, Saskatchewan\. His brother Dustin was drafted\.</information\><think\>Doc 2 says Chris Pronger planned to join his older brother Sean at Bowling Green State University\. Therefore, Sean James Pronger’s younger brother is Chris Pronger\. Now I need to find which hockey team Chris Pronger currently works for\.</think\><search\>Chris Pronger current team</search\><information\>Doc 1 \(Title: "Chris Pronger"\)Chris Pronger Christopher Robert Pronger \(born October 10, 1974\) is a Canadian former professional ice hockey defenceman who is currently the senior advisor of hockey operations for the Florida Panthers\. He had not played since November 2011 due to post\-concussion syndrome\.Doc 2 \(Title: "Chris Pronger"\)Chris Pronger Christopher Robert Pronger is a Canadian former professional ice hockey defenceman who is currently the senior advisor of hockey operations for the Florida Panthers\.</information\><think\>The second search confirms that Chris Pronger currently works as senior advisor of hockey operations for the Florida Panthers\. Therefore, the hockey team is the Florida Panthers\.</think\><answer\>Florida Panthers</answer\>Table 11:Case Study: Multi\-Hop QA\.