When the Judge Should Not Decide: Evidence-Locked, Non-Compensatory Selection Bounds LLM-Judge Failure in Reasoning Pipelines

# When the Judge Should Not Decide: Evidence-Locked, Non-Compensatory Selection Bounds LLM-Judge Failure in Reasoning Pipelines
Source: [https://arxiv.org/html/2608.07813](https://arxiv.org/html/2608.07813)
\\workshoptitle

Can We Trust the Judge? Building Reliable Evaluation for Language Models

Yiyao Zhanga, Diksha Goelb, Hussain Ahmadc, Shixun Huanga, Jun Shena aSchool of Computing and Information Technology, University of Wollongong, Wollongong, NSW, Australia bCSIRO’s Data61, Clayton, VIC, Australia cSchool of Computer Science and Information Technology, Adelaide University, Adelaide, SA, Australia Author e\-mail addresses:yiyao\.zhang@uow\.edu\.au \(Y\. Zhang\), diksha\.goel@csiro\.au \(D\. Goel\), hussain\.ahmad@adelaide\.edu\.au \(H\. Ahmad\), shixun\_huang@uow\.edu\.au \(S\. Huang\), jshen@uow\.edu\.au \(J\. Shen\)

###### Abstract

An LLM judge deployed inside a reasoning pipeline does not merely measure quality , it decides which answer ships\. We show that the cost of that decision depends less on judge accuracy than on the*decision rule*the judge is embedded in\. On frozen candidate pools from four GRPO policies, an unconstrained scalar DeepSeek\-R1\-7B judge buys almost nothing over answer\-level majority vote \(\+1\.0\+1\.0pp on 500 GSM8K questions,\+0\.34\+0\.34EM on 300 HotpotQA questions\), and on a frozen\-rule 30\-question confirmation split it is*10 points worse*than majority , a judge that destroys accuracy while scoring candidates confidently\. We then subordinate the same judge to Evidence\-Locked Derive–Gate–Repair \(EL\-DGR\), a task\-adaptive non\-compensatory rule under which a judge preference may override evidence\-supported consensus only with an extractive evidence certificate, and a repair only when neither alternative is certified and the repair is\. With no change to the judge, the candidates, or the budget,EL\-DGRreaches 58\.2% on GSM8K \(vs\. 56\.8% judge, 55\.8% majority, 55\.4% first candidate\) and 17\.33 EM / 25\.46 F1 on HotpotQA \(vs\. 15\.67/23\.49, 15\.33/23\.19, 15\.33/22\.97\), improving on first\-candidate GRPO by\+2\.8\+2\.8pp \(exact McNemarp=0\.0026p\{=\}0\.0026\) and\+2\.00\+2\.00EM \(p=0\.070p\{=\}0\.070, borderline\)\. A decision audit shows why:EL\-DGRoverturns consensus on only 8 of 30 pilot questions and never converts a correct consensus into an incorrect answer\. We also report what did*not*work: the same seven\-channel decomposition used as a step\-level gated training reward is null, and corrected channel\-drop ablations show no channel is individually necessary \(p=1\.0p\{=\}1\.0throughout\)\. The practitioner\-facing finding is negative about judges and positive about admissibility , bound the judge’s blast radius rather than trying to make it accurate\.

## 1Introduction

LLM judges are increasingly load\-bearing\. They gate which ofGGsampled candidates is returned, filter preference data, and supply reward for RL post\-training\(Zhenget al\.,[2023](https://arxiv.org/html/2608.07813#bib.bib71); Ouyang and others,[2022](https://arxiv.org/html/2608.07813#bib.bib2); Shao and others,[2024](https://arxiv.org/html/2608.07813#bib.bib5); Lightman and others,[2023](https://arxiv.org/html/2608.07813#bib.bib4)\)\. In each case the judge’s output is not a measurement that a human later reads , it is a decision that the pipeline executes\. The same is true wherever an LLM score is wired into an operational loop: multi\-LLM scoring and selection for portfolio construction\(Chenet al\.,[2025](https://arxiv.org/html/2608.07813#bib.bib77)\), agentic triage and remediation of software vulnerabilities\(Arifinet al\.,[2026](https://arxiv.org/html/2608.07813#bib.bib79)\), and clinical question answering, where model\-to\-model disagreement is large enough that the aggregation rule is itself a safety\-relevant design choice\(Santhoshet al\.,[2026](https://arxiv.org/html/2608.07813#bib.bib78)\)\. Evaluation validity is therefore not a property of the judge alone but of the judge*plus*the rule that consumes its score\. A judge with a 60% pairwise agreement rate is harmless if it can only break ties among already\-admissible answers, and dangerous if it can unilaterally overwrite a correct consensus\.

Most work on judge reliability targets the first half of that pair , measuring and correcting the judge itself, through position\-bias audits and swap averaging\(Wanget al\.,[2024](https://arxiv.org/html/2608.07813#bib.bib72)\), self\-preference detection\(Panicksseryet al\.,[2024](https://arxiv.org/html/2608.07813#bib.bib73)\), and the calibration and agreement protocols surveyed byGuet al\.\([2024](https://arxiv.org/html/2608.07813#bib.bib74)\)\. We study the second half: holding the judge, candidate pool, prompts, and compute budget fixed and varying only the decision rule isolates a quantity judge benchmarks do not report , how much of a judge’s damage comes from its scores, and how much from the authority it is granted\.

#### The conflation a scalar judge cannot see\.

Consider*“What is the capital of the country that hosted the 2022 FIFA World Cup?”*and the chain: \(1\)*The 2022 FIFA World Cup was hosted by Qatar\.*\(2\)*The capital of Qatar is Dubai\.*\(3\)*Therefore the answer is Dubai\.*Step 3 follows validly from Step 2 and the chain reads fluently, but Step 2 is contradicted by the first retrievable document\. A judge emitting one scalar for “quality” scores this chain highly, because logical structure and evidential support have been collapsed into a number that cannot represent their disagreement , and the failure is silent, since the score carries no indication of*which*dimension produced it\.

#### Two responses, one of which fails\.

The obvious fix is to decompose: score the step on several explicit dimensions and recombine\. We implemented this asRA\-MSD\(Retrieval\-Augmented Multi\-Signal Decision reward\): seven step\-level channels with logic and factuality gating the weighted auxiliary score \(Section[4](https://arxiv.org/html/2608.07813#S4)\)\. As a*training*signal this did not work , held\-out accuracy was indistinguishable from plain GRPO, and channel\-drop ablations show no channel is individually necessary \(Section[7](https://arxiv.org/html/2608.07813#S7)\)\. We report it because it localizes the problem: any scheme that eventually recombines the channels into one scalar reinherits the conflation it was built to remove\.

The response that does work applies the decomposition elsewhere in the pipeline: not as a summand, but as an*admissibility constraint*on the judge’s authority\. Evidence\-Locked Derive–Gate–Repair \(EL\-DGR\) partitions candidates into feasibility strata by a cheap certificate , an extractive occurrence check against retrieved evidence for retrieval QA, an independently derived and re\-checked numeric answer for arithmetic\. The judge’s scalar preference then ranks candidates*within*a stratum but can never promote one across strata, so an evidence\-supported consensus answer cannot be displaced by a higher\-scoring but uncertified alternative\.

#### Contributions\.

1. 1\.A measurement of judge authority, not judge accuracy:on matched frozen pools, an unconstrained scalar judge is worth\+1\.0\+1\.0pp \(GSM8K,n=500n\{=\}500\) and\+0\.34\+0\.34EM \(HotpotQA,n=300n\{=\}300\) over majority vote, and−10\.0\-10\.0pp on a frozen\-rule confirmation split \(n=30n\{=\}30\), below even first\-candidate selection \(Section[6](https://arxiv.org/html/2608.07813#S6)\)\.
2. 2\.EL\-DGR, a non\-compensatory decision rulethat subordinates the same judge to task\-adaptive evidence certificates, improving every protocol\-matched selector on both datasets with no extra candidates and≤5\\leq 5s/question of verifier latency \(Sections[4](https://arxiv.org/html/2608.07813#S4)–[6](https://arxiv.org/html/2608.07813#S6)\)\.
3. 3\.A decision audit and a negative result:whereEL\-DGR’s gain comes from \(8/30 consensus overrides, 0 correct\-to\-incorrect flips\) and where the same decomposition fails as a training reward \(Sections[6\.3](https://arxiv.org/html/2608.07813#S6.SS3)–[7](https://arxiv.org/html/2608.07813#S7)\)\.

This is a post\-training*selection*study: we do not claimEL\-DGRtrains a better policy, and absolute scores are not comparable to published state of the art, since the candidate artifacts come from a 3B policy under a constrained budget\. All selectors consume the identical frozen pool, so the paired contrasts are internally valid\.

## 2Related Work

#### Judge reliability\.

The dominant framing treats evaluator quality as a measurement problem: how well does the judge agree with humans, and which biases distort that agreement? This line establishes pairwise\-agreement benchmarks\(Zhenget al\.,[2023](https://arxiv.org/html/2608.07813#bib.bib71)\), documents position and ordering effects with swap\-averaging remedies\(Wanget al\.,[2024](https://arxiv.org/html/2608.07813#bib.bib72)\), and shows that evaluators prefer their own generations\(Panicksseryet al\.,[2024](https://arxiv.org/html/2608.07813#bib.bib73)\);Guet al\.\([2024](https://arxiv.org/html/2608.07813#bib.bib74)\)survey the resulting protocols\. We take the judge’s error profile as given , neither debiasing nor fine\-tuning it , and ask what the surrounding decision rule does to the*consequences*of those errors\. That second factor can dominate the first: below, identical scores swing from 10 points below majority vote to the best selector tested, purely through admissibility\. Constraining authority rather than improving the scorer is the same move made by compositional shielding, where a verifier decides which agent actions are admissible and the system abstains when none is certified\(Zhanget al\.,[2026b](https://arxiv.org/html/2608.07813#bib.bib75)\), and by uncertainty\-gated meta\-reasoning, where a competence test decides whether an auxiliary call is consulted at all\(Zhanget al\.,[2026a](https://arxiv.org/html/2608.07813#bib.bib76)\)\.EL\-DGRapplies that discipline to the narrower object of a judge’s selection authority over a frozen pool\.

#### Step supervision and grounded verification\.

Process reward models supply dense local feedback but usually collapse logical validity, factual support, and redundancy into one scalar\(Lightman and others,[2023](https://arxiv.org/html/2608.07813#bib.bib4); Wang and others,[2025](https://arxiv.org/html/2608.07813#bib.bib22)\)\. Retrieval\-based systems use external evidence during*generation*\(Lewis and others,[2020](https://arxiv.org/html/2608.07813#bib.bib17); Yaoet al\.,[2023](https://arxiv.org/html/2608.07813#bib.bib27); Asaiet al\.,[2024](https://arxiv.org/html/2608.07813#bib.bib25); Nakanoet al\.,[2021](https://arxiv.org/html/2608.07813#bib.bib26)\), whereas we use retrieval at scoring time\. The grounding channel is closest to claim\-verification metrics such as FActScore\(Minet al\.,[2023](https://arxiv.org/html/2608.07813#bib.bib24); Chernet al\.,[2023](https://arxiv.org/html/2608.07813#bib.bib42)\), moved from offline evaluation into an online decision rule; self\-consistency\(Wanget al\.,[2023](https://arxiv.org/html/2608.07813#bib.bib29)\)supplies our judge\-free majority baseline\.

#### Positioning against recent multi\-signal methods\.

Table[5](https://arxiv.org/html/2608.07813#A1.T5)states the comparison boundary against five 2025 systems that also decompose or densify reasoning feedback \(Table[5](https://arxiv.org/html/2608.07813#A1.T5), Appendix[A](https://arxiv.org/html/2608.07813#A1)\): DRM\(Wanget al\.,[2025](https://arxiv.org/html/2608.07813#bib.bib66)\), BCRL\(Wu and others,[2025](https://arxiv.org/html/2608.07813#bib.bib67)\), GAR\(Liuet al\.,[2025](https://arxiv.org/html/2608.07813#bib.bib68)\), AutoDSPy\(Azimet al\.,[2025](https://arxiv.org/html/2608.07813#bib.bib69)\), and SWiRL\(Luet al\.,[2025](https://arxiv.org/html/2608.07813#bib.bib70)\)\. Their published scores are not copied into Table[2](https://arxiv.org/html/2608.07813#S6.T2): model, retrieval corpus, budget, and protocol all differ\. Our matched baselines are therefore*selectors*that consume exactly the same frozen candidate pool , the only class for which a paired test on a fixed pool is meaningful\.

## 3Setting: The Judge as a Selector

#### Judge conflation\.

Let𝒟t\\mathcal\{D\}\_\{t\}be the documents retrieved for a reasoning stepsts\_\{t\}and letfNLI​\(st∣d\)∈\[−1,1\]f\_\{\\mathrm\{NLI\}\}\(s\_\{t\}\\mid d\)\\in\[\-1,1\]map contradiction to−1\-1, neutral to0, and entailment to\+1\+1\. A step\-level scoreRRsuffers*\(η,δ\)\(\\eta,\\delta\)\-conflation*, forδ∈\(0,1\]\\delta\\in\(0,1\]andη<0\\eta<0, if some reachable step satisfies

R​\(st\)≥δandmaxd∈𝒟t⁡fNLI​\(st∣d\)≤η\.R\(s\_\{t\}\)\\geq\\delta\\quad\\text\{and\}\\quad\\max\_\{d\\in\\mathcal\{D\}\_\{t\}\}f\_\{\\mathrm\{NLI\}\}\(s\_\{t\}\\mid d\)\\leq\\eta\.\(1\)The Qatar/Dubai step is a severe\(η=−0\.8,δ=0\.85\)\(\\eta\{=\}\{\-\}0\.8,\\delta\{=\}0\.85\)\-conflation\. Closed\-book judges admit conflation by construction: nothing in their input distinguishes a supported step from a fluent unsupported one\. Outcome reward models admit it at every intermediate step; rule\-based format\-plus\-match rewards admit it whenever surface form and evidence support decorrelate\.

#### Selection protocol\.

Fix a questionQQ, a frozen candidate set\{y1,…,yG\}\\\{y\_\{1\},\\dots,y\_\{G\}\\\}with normalized answersaja\_\{j\}, and retrieved evidenceEE\. Letmmbe the answer\-level majority candidate\(Wanget al\.,[2023](https://arxiv.org/html/2608.07813#bib.bib29)\),ssthe candidate a scalar judge prefers, andrran independently derived and checked repair\. A*selector*maps\(Q,\{yj\},E\)\(Q,\\\{y\_\{j\}\\\},E\)to a returned answer\. Our baselines are first candidate \(what a single\-sample pipeline returns\), majority vote \(judge\-free consensus\), and the scalar judgess\(the judge given full authority\);EL\-DGRis a fourth selector using the samessandEE\. Because every selector is fed the identical pool, differences measure decision rules, not generation\.

## 4Method

### 4\.1Seven diagnostic channels

Figure[1](https://arxiv.org/html/2608.07813#S4.F1)shows the pipeline: a frozen candidate group and the task input enter a judge that scores each candidate on several explicit aspects rather than emitting one number, and the scores are consumed by a non\-compensatory rule that decides what is*admissible*before deciding what is best\. Everything left of the decision rule is standard; the contribution is the box on the right, and specifically the order in which its cases are tried\.

![Refer to caption](https://arxiv.org/html/2608.07813v1/figures/eldgr_pipeline.png)Figure 1:The judge\-as\-selector pipeline\. Candidates and task input are scored by an LLM judge on seven diagnostic aspects \(Table[1](https://arxiv.org/html/2608.07813#S4.T1)\) instead of a single scalar; an evidence\-locked rule then partitions candidates into feasibility strata, the judge’s preference ranks candidates only within a stratum, and a checked repair is admitted only when no certified candidate exists\. The exact ordering is Eq\. \([3](https://arxiv.org/html/2608.07813#S4.E3)\)\.RA\-MSDreplaces one scalar with seven channels, each scored in\[0,1\]\[0,1\]and each targeting a failure mode a scalar judge cannot separate \(Table[1](https://arxiv.org/html/2608.07813#S4.T1)\)\. Grounding channels follow claim\-verification metrics\(Minet al\.,[2023](https://arxiv.org/html/2608.07813#bib.bib24)\)but are computed online at reward time rather than offline\. Channels are*complementary*, not orthogonal: they target different failures but are not assumed statistically independent\. The point of the decomposition is auditability , for any step, the seven values are a trace of*why*it scored as it did\.

Two channels carry the structural load\.Logical validityRlogicR\_\{\\mathrm\{logic\}\}asks whether a step is inferentially connected to what precedes it, independently of whether its content is true , a step can follow correctly from false premises, exactly the Qatar/Dubai pattern\.Factual groundingRfactR\_\{\\mathrm\{fact\}\}asks the complementary question, whether the step survives contact with retrieved evidence, and is the channel Eq\. \([1](https://arxiv.org/html/2608.07813#S3.E1)\) makes necessary\. Among the supporting five, triangulation requiresM=3M\{=\}3distinct corroborating sources so one topically\-overlapping document cannot certify a claim; adversarial robustness comparesRfactR\_\{\\mathrm\{fact\}\}against a counterfactually perturbed step, catching lexical rather than semantic support; and minimal sufficiency closes the reward\-hacking route of padding a step with unverifiable material\.

Table 1:The sevenRA\-MSDchannels, each computed from step text, reasoning context, and retrieved evidence𝒟t\\mathcal\{D\}\_\{t\}, and each isolating a judge failure mode a single scalar cannot represent\.ctc\_\{t\}: token confidence;ete\_\{t\}: evidence strength;nindepn\_\{\\text\{indep\}\}: distinct corroborating sources \(M=3M\{=\}3\);nnewn\_\{\\text\{new\}\}: new unsupported claims \(β=0\.1\\beta\{=\}0\.1\)\.
### 4\.2Step\-level gating \(the variant that failed\)

LetR¯​\(σt\)=∑i=17wi​Ri​\(σt\)\\bar\{R\}\(\\sigma\_\{t\}\)=\\sum\_\{i=1\}^\{7\}w\_\{i\}R\_\{i\}\(\\sigma\_\{t\}\)with∑iwi=1\\sum\_\{i\}w\_\{i\}=1and𝐰=\(\.20,\.20,\.10,\.15,\.10,\.15,\.10\)\\mathbf\{w\}=\(\.20,\.20,\.10,\.15,\.10,\.15,\.10\)\. To block compensation, logic and factuality act as feasibility constraints with thresholdτ=0\.5\\tau=0\.5:

Rstep​\(σt\)=\{R¯​\(σt\),min⁡\{Rlogic​\(σt\),Rfact​\(σt\)\}≥τ,min⁡\{Rlogic​\(σt\),Rfact​\(σt\)\},otherwise,R\_\{\\mathrm\{step\}\}\(\\sigma\_\{t\}\)=\\begin\{cases\}\\bar\{R\}\(\\sigma\_\{t\}\),&\\min\\\{R\_\{\\mathrm\{logic\}\}\(\\sigma\_\{t\}\),R\_\{\\mathrm\{fact\}\}\(\\sigma\_\{t\}\)\\\}\\geq\\tau,\\\\ \\min\\\{R\_\{\\mathrm\{logic\}\}\(\\sigma\_\{t\}\),R\_\{\\mathrm\{fact\}\}\(\\sigma\_\{t\}\)\\\},&\\text\{otherwise\},\\end\{cases\}\(2\)and the completion reward isRtotal​\(y\)=3\.0​RORM\+0\.5​Rformat\+1n​∑tRstep​\(σt\)R\_\{\\mathrm\{total\}\}\(y\)=3\.0\\,R\_\{\\mathrm\{ORM\}\}\+0\.5\\,R\_\{\\mathrm\{format\}\}\+\\frac\{1\}\{n\}\\sum\_\{t\}R\_\{\\mathrm\{step\}\}\(\\sigma\_\{t\}\), optimized with TRL’sGRPOTrainer; baselines differ only in the reward callback\. Equation \([2](https://arxiv.org/html/2608.07813#S4.E2)\) prevents*step\-level*compensation but still collapses all feasible channels into one number before the optimizer sees them; empirically it is null \(Section[7](https://arxiv.org/html/2608.07813#S7)\), motivating moving the constraint from the summand to the decision\.

### 4\.3Evidence\-Locked Derive–Gate–Repair

EL\-DGRmakes evidence an*admissibility*condition at the group level\. For retrieval QA define the strict extractive certificatec​\(a,E\)=𝟏​\[a≠∅∧a​occurs verbatim in​E\]c\(a,E\)=\\mathbf\{1\}\[a\\neq\\varnothing\\wedge a\\ \\text\{occurs verbatim in\}\\ E\]and return

aret=\{r,c​\(r,E\)=1,c​\(m,E\)=c​\(s,E\)=0,s,c​\(s,E\)=1,c​\(m,E\)=0,m,otherwise\.a\_\{\\mathrm\{ret\}\}=\\begin\{cases\}r,&c\(r,E\)=1,\\ c\(m,E\)=c\(s,E\)=0,\\\\ s,&c\(s,E\)=1,\\ c\(m,E\)=0,\\\\ m,&\\text\{otherwise\}\.\\end\{cases\}\(3\)For arithmetic the constraint is executable rather than documentary: the verifier must expose a derived numeric answer together with a consistency check, and the branch returns the checked repairrrwhen it parses andmmotherwise\. The task\-adaptive rule is

aEL\-DGR=\{aarith,τ​\(Q\)=arithmetic,aret,τ​\(Q\)=retrieval,a\_\{\\textsc\{EL\-DGR\}\}=\\begin\{cases\}a\_\{\\mathrm\{arith\}\},&\\tau\(Q\)=\\mathrm\{arithmetic\},\\\\ a\_\{\\mathrm\{ret\}\},&\\tau\(Q\)=\\mathrm\{retrieval\},\\end\{cases\}\(4\)withτ\\taufixed from the benchmark task type, never selected using test answers\.

#### Why the ordering is the whole method\.

Equation \([3](https://arxiv.org/html/2608.07813#S4.E3)\) is deliberately asymmetric: consensusmmis the default and is displaced only by a*certified*alternative; the judge’s preferencessis admitted only when certified*and*consensus is not; a repairrronly when neithermmnorssis certified andrritself is\. Auxiliary scores , all seven channels and the judge’s scalar , rank candidates only inside a feasibility stratum\. This bounds the judge’s error profile structurally: to cause a regression the judge must prefer an answer that is both wrong and verbatim\-present in the evidence while the correct consensus is absent from it\. The certificate is cheap and conservative; it rejects valid paraphrases, which costs recall, and the arithmetic branch sidesteps that by checking an executable derivation instead of document occurrence\. Eq\. \([3](https://arxiv.org/html/2608.07813#S4.E3)\) also induces a lexicographic group rank usable for training \(certified≻\\succuncertified; supported consensus breaks the first tie\), but we evaluate it only as a post\-training selector here\.

### 4\.4What the decomposition does and does not buy

Three component\-level results characterize the design choices above\. All describe properties of the*score*; none is a policy\-improvement certificate, which matters because Section[7](https://arxiv.org/html/2608.07813#S7)reports a null training result\. For the pre\-gate weighted score, bilinearity gives

Var​\(R¯\)=∑i=17wi2​Var​\(Ri\)\+2​∑1≤i<j≤7wi​wj​Cov​\(Ri,Rj\),\\mathrm\{Var\}\(\\bar\{R\}\)=\\sum\_\{i=1\}^\{7\}w\_\{i\}^\{2\}\\mathrm\{Var\}\(R\_\{i\}\)\+2\\\!\\\!\\sum\_\{1\\leq i<j\\leq 7\}\\\!\\\!w\_\{i\}w\_\{j\}\\mathrm\{Cov\}\(R\_\{i\},R\_\{j\}\),\(5\)so any pair withwi​wj\>0w\_\{i\}w\_\{j\}\>0andCov​\(Ri,Rj\)<0\\mathrm\{Cov\}\(R\_\{i\},R\_\{j\}\)<0strictly lowersVar​\(R¯\)\\mathrm\{Var\}\(\\bar\{R\}\)\. The anti\-conflation design*manufactures*that negative covariance: conflated steps in the sense of Eq\. \([1](https://arxiv.org/html/2608.07813#S3.E1)\) have highRlogicR\_\{\\mathrm\{logic\}\}and lowRfactR\_\{\\mathrm\{fact\}\}by construction, so channel disagreement on exactly the cases of interest reduces rather than amplifies variance\. Two caveats bound the claim\. The deployed gate is nonlinear, so Eq\. \([5](https://arxiv.org/html/2608.07813#S4.E5)\) does*not*describeRstepR\_\{\\mathrm\{step\}\}; and the induced score\-function bound𝔼​‖gt‖2≤G2​𝔼​\[R¯2\]\\mathbb\{E\}\\\|g\_\{t\}\\\|^\{2\}\\leq G^\{2\}\\mathbb\{E\}\[\\bar\{R\}^\{2\}\]\(under aGG\-Lipschitz log\-policy\) constrains one factor of a second moment, not learning stability\. Two smaller results follow similarly:RfactR\_\{\\mathrm\{fact\}\}is a monotone transform of the statistic defining the unsupported\-step indicatorUtU\_\{t\}, so penalizing it penalizes𝔼​\[Ut\]\\mathbb\{E\}\[U\_\{t\}\]*conditional on the corpus*; andRepist=1−\|ct−et\|R\_\{\\mathrm\{epist\}\}=1\-\|c\_\{t\}\-e\_\{t\}\|, an absolute\-deviation loss, elicits the median of the model’s evidence\-strength belief rather than its full distribution \(Appendix[B](https://arxiv.org/html/2608.07813#A2)\)\. Nothing here implies that optimizingRstepR\_\{\\mathrm\{step\}\}improves a policy , the analysis characterizes a random variable, while the empirical failure is about what a*scalarized*score transmits through GRPO\.EL\-DGRsidesteps that gap by never scalarizing across feasibility strata\.

## 5Experimental Setup

#### Policies and data\.

Base policyQwen2\.5\-3B\-Instructwith LoRA \(r=16r\{=\}16,α=32\\alpha\{=\}32onq,k,v,o\_proj;≈\\approx4\.7M trainable parameters\), lr2×10−52\\times 10^\{\-5\}, seed 42, 128 new tokens per completion\. GSM8K\(Cobbe and others,[2021](https://arxiv.org/html/2608.07813#bib.bib8)\): train 500, evaluate 500 held out\. HotpotQA\(Yang and others,[2018](https://arxiv.org/html/2608.07813#bib.bib9)\)distractor: train 500, evaluate 300 held out\. Retrieval is BM25 over the supporting\-facts corpus \(K=5K\{=\}5\) for HotpotQA and over 1,000 training solutions for GSM8K, where grounding therefore measures consistency with alternative valid solution paths rather than external knowledge\. NLI throughout iscross\-encoder/nli\-deberta\-v3\-small\.

#### Frozen candidate pool, judge, and statistics\.

One pool per question is built from the four seed\-42 full\-budget policy artifacts \(GRPO, ORM, PRM,RA\-MSD\), so every selector sees exactly the same candidates\. The judge is a localdeepseek\-r1:7bvia Ollama with structured JSON output and hidden thinking disabled; it receives the HotpotQA distractor context and*never*the gold answer, and homogeneous groups are returned without a judge call\. We report percentile bootstrap intervals over 20,000 paired question\-level resamples and two\-sided exact McNemar tests; these quantify evaluation\-set uncertainty only, not training\-seed variance\. HotpotQA uses official normalization, reporting EM and token F1\.

## 6Results

### 6\.1Matched full\-budget selection

Table 2:Full\-budget matched post\-training selection \(GSM8Kn=500n\{=\}500, HotpotQAn=300n\{=\}300\); all selectors consume the identical frozen candidate pool\.Δ\\DeltaisEL\-DGRminus that row, in percentage points with paired bootstrap 95% intervals \(20,000 resamples\);ppis the two\-sided exact McNemar test on EM\. BoldΔ\\Deltamarks contrasts resolved atα=0\.05\\alpha\{=\}0\.05\.Two things stand out in Table[2](https://arxiv.org/html/2608.07813#S6.T2)\. First,full judge authority is worth very little: the scalar judge beats judge\-free majority vote by1\.01\.0pp on GSM8K and0\.340\.34EM on HotpotQA, well inside noise\. Second,the same judge underEL\-DGRis worth substantially more:\+2\.4\+2\.4pp and\+2\.0\+2\.0EM over majority, from identical scores and identical candidates\. The difference is attributable entirely to what the judge may overrule\.

On GSM8K the arithmetic derive–repair branch beats first\-candidate by2\.82\.8pp with 17 paired wins and 3 losses \(exact McNemarp=0\.0026p\{=\}0\.0026\); the contrast against the scalar judge is positive but unresolved \(\+1\.4\+1\.4pp,\[−0\.2,3\.0\]\[\-0\.2,3\.0\],p=0\.143p\{=\}0\.143\)\. This supports*executable task constraints*, not retrieval\-channel importance , GSM8K has no external facts to certify\. On HotpotQA, evidence locking gains2\.02\.0EM and2\.502\.50F1 with seven EM wins and one loss: the bootstrap interval excludes zero, but the conservative exact test is borderline \(p=0\.0703p\{=\}0\.0703\)\. F1 gains follow the same ordering\.EL\-DGRis positive against all three matched selectors on both datasets, and under\-powered only against the stronger judge\.

### 6\.2Frozen\-rule confirmation: a judge that is worse than no judge

The rule in Section[4\.3](https://arxiv.org/html/2608.07813#S4.SS3)was developed on three disjoint HotpotQA subsets totalling 120 questions; we froze the prompts, certificate, and decision order, then evaluated once on a fourth 30\-question subset \(seed 2029\) disjoint from all development data, with the corrected seed\-42 GRPO checkpoint producing the same four stochastic candidates \(temperature0\.70\.7, top\-pp0\.90\.9\) for every selector\.

Table 3:Frozen\-rule HotpotQA confirmation split \(n=30n\{=\}30\), evaluated once after the decision rule was locked\. The unconstrained scalar judge is the*worst*selector tested , below majority vote and below taking the first sample\.Table[3](https://arxiv.org/html/2608.07813#S6.T3)is the result we consider most relevant to judge reliability\. Given full authority over a four\-candidate pool, the scalar judge scores10\.010\.0EM ,1010points below judge\-free majority vote and3\.33\.3below returning the first sample\. It is not merely uninformative but actively harmful, and harmful*confidently*, emitting well\-formed JSON preferences on every question\. Restricted by Eq\. \([3](https://arxiv.org/html/2608.07813#S4.E3)\) to certified overrides, the same judge yields the best selector in the table \(26\.726\.7EM,\+16\.7\+16\.7over its unconstrained self, exactp=0\.0625p\{=\}0\.0625\)\. Against majority the\+6\.7\+6\.7pp difference is unresolved atn=30n\{=\}30\(p=0\.5p\{=\}0\.5\): a frozen\-rule confirmation thatEL\-DGRimproved every tested selector on this split, not population\-level superiority\.

### 6\.3Decision audit: where the gain comes from

On the 30 confirmation questionsEL\-DGRreturns the majority answer on22, the evidence\-certified judge choice on4, and an evidence\-certified repair on4, changingnocorrect majority decision into an incorrect one and adding two exact matches\. This is the intended shape of a bounded judge: it declines to act on 73% of questions, confining the judge’s influence to cases where consensus carries no evidential support\. An unconstrained judge can convert correct consensus into error anywhere; under Eq\. \([3](https://arxiv.org/html/2608.07813#S4.E3)\) only where evidence contains the wrong answer and not the right one\.

#### Cost\.

Excluding candidate generation, common to all selectors, the scalar and derive–gate–repair calls average1\.771\.77s and2\.912\.91s per pilot question \(4\.684\.68s total\); at full budget the pair averages1\.361\.36s per GSM8K and4\.994\.99s per HotpotQA question, with verifier work skipped entirely for 352 of 500 homogeneous GSM8K groups \(172 of 300 HotpotQA groups are retrospectively homogeneous and skippable in deployment\)\. This is an inference\-time accuracy–latency trade\-off, not a free lunch\.

## 7What Did Not Work

#### The gated reward is null\.

Training with Eq\. \([2](https://arxiv.org/html/2608.07813#S4.E2)\) did not improve held\-out accuracy over plain GRPO\. Figure[2](https://arxiv.org/html/2608.07813#S7.F2)shows why we decline to read anything into small differences between reward methods: the on\-policy trajectories \(two sampled completions per optimizer step, 10\-step rolling mean\) fluctuate substantially on GSM8K , including a transient GRPO peak near step 72 , and are sparse and frequently zero on HotpotQA\. No method separates stably, which means final\-checkpoint selection alone can manufacture or erase a small reported gap\. UnderG=2G\{=\}2the z\-normalized advantages are also close to binary, which plausibly contributes\.

![Refer to caption](https://arxiv.org/html/2608.07813v1/x1.png)Figure 2:Sampled\-completion exact match during training \(seed 42, two candidates per step, 10\-step rolling mean\) , on\-policy diagnostics,*not*held\-out accuracy\. No reward method separates stably, the basis for treating the gated\-reward comparison as null rather than as a small win or loss\.
#### The legacy ablation collapses\.

Single\-seed GSM8K channel\-drop runs fall to44\.644\.6–47\.4%47\.4\\%: not just below fullRA\-MSD\(55\.6%55\.6\\%\) but88–1111pp below GRPO, which uses none of the channels \(p<3\.4×10−6p<3\.4\\times 10^\{\-6\}on every row\)\. A collapse of this shape is incompatible with a marginal\-importance reading; zeroing and renormalizing one weight perturbed the trajectory under a high\-varianceG=2G\{=\}2single\-seed protocol\. We retain it as a diagnostic failure, not evidence that each channel matters\.

#### Corrected ablations show no channel is necessary\.

Rerunning every channel drop on HotpotQA with the corrected chat template,G=4G\{=\}4, and 30 optimizer steps \(Table[4](https://arxiv.org/html/2608.07813#S7.T4)\) removes the pathology , all rows land between 18% and 20% EM , but supplies no support for the decomposition either\. Dropping logic or factuality removes the channel from both the weighted score and the feasibility gate; other drops zero and renormalize the corresponding auxiliary weight\. No drop differs from full gatedRA\-MSDby more than one example, every exact paired McNemar test against full givesp=1\.0p\{=\}1\.0, and the factuality, information, and minimality drops are numerically*above*full\. We therefore cannot claim that any individual channel is load\-bearing at this budget; multi\-seed, full\-budget ablations remain required\.

Table 4:Corrected HotpotQA channel\-drop pilot \(seed 42,G=4G\{=\}4, 30 steps, same 100 evaluation examples for all rows\)\. Deltas are EM percentage points\. Every drop vs\. full: exact McNemarp=1\.0p\{=\}1\.0\.The boundary these draw is the point of the paper: the decomposition earns nothing when summed into a training signal, and something when used as a*feasibility partition*, because a partition cannot be summed away\.

## 8Discussion and Limitations

#### Bound the judge, don’t perfect it\.

Tables[2](https://arxiv.org/html/2608.07813#S6.T2)and[3](https://arxiv.org/html/2608.07813#S6.T3)say that judge*accuracy*and judge*usefulness*are only loosely coupled: the samedeepseek\-r1:7bscores are worth−10\-10pp or\+6\.7\+6\.7pp against majority vote depending only on the rule that consumes them\. Before investing in a better judge, ask what the current one may overrule , a design parameter under direct control\. A judge deployment record should state \(i\) the default the judge must beat, \(ii\) the certificate required to beat it, and \(iii\) the measured override rate split by outcome; ours are 22/30 defaults preserved, extractive occurrence, and 8 overrides with 0 correct\-to\-incorrect flips\.

#### Reading the null result correctly\.

Section[7](https://arxiv.org/html/2608.07813#S7)is not evidence that decomposed evaluation does not help: the decomposition failed where it was summed back into a scalar and succeeded where it was used to partition\. A judge pipeline that computes seven diagnostics and then averages them has, from the optimizer’s perspective, computed one number\.

#### Limitations\.

*Verifier\-relative factuality:*RfactR\_\{\\mathrm\{fact\}\}and the certificate are defined against the retrieved corpus and NLI model, so their errors propagate into the decision\.*Certificate scope:*exact\-substring matching rejects supported paraphrases and does not prove the extracted span satisfies the question’s relation\.*Statistical power:*one training seed and one candidate pool; the HotpotQA first\-candidate contrast is borderline \(p=0\.0703p\{=\}0\.0703\) andn=30n\{=\}30cannot resolve the majority contrast\.*Scale:*one 3B policy, one 7B judge, two benchmarks, with candidate artifacts inheriting a legacy missing\-chat\-template defect, so absolute scores are not comparable to published state of the art\.*Responsible use:*evidence\-aware selection is more persuasive and inherits corpus bias; high\-stakes deployments should retain evidence logs and human review\.

## 9Conclusion

On frozen candidate pools, an unconstrained scalar judge is worth roughly nothing over majority vote at full budget and is the worst selector we tested on a frozen\-rule split; the identical judge under task\-adaptive evidence certificates is the best \(58\.2% on GSM8K,p=0\.0026p\{=\}0\.0026; 17\.33/25\.46 EM/F1 on HotpotQA,p=0\.0703p\{=\}0\.0703\) with zero correct\-to\-incorrect flips\. A judge’s error profile is bounded far more cheaply by constraining what it may overrule than by improving what it scores\. Multi\-seed, pre\-registered replication is the next step\.

## References

- AgenticVM: agentic AI for adaptive software vulnerability management\.IEEE Software\.Cited by:[§1](https://arxiv.org/html/2608.07813#S1.p1.1)\.
- A\. Asai, Z\. Wu, Y\. Wang, A\. Sil, and H\. Hajishirzi \(2024\)Self\-rag: learning to retrieve, generate, and critique through self\-reflection\.Note:arXiv:2310\.11511External Links:2310\.11511Cited by:[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px2.p1.1)\.
- N\. Azim, A\. U\. Alam, H\. B\. Omar, A\. M\. M\. A\. Jami, J\. I\. Ahad, M\. R\. Kabir, Md\. I\. Hossain, F\. Rahman, M\. R\. Amin, S\. Rahman, and N\. Mohammed \(2025\)AutoDSPy: automating modular prompt design with reinforcement learning for small and large language models\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track,pp\. 2881–2896\.Cited by:[Table 5](https://arxiv.org/html/2608.07813#A1.T5.3.5.4.1),[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px3.p1.1)\.
- K\. Chen, H\. Ahmad, D\. Goel, and C\. Szabo \(2025\)3S\-Trader: a multi\-LLM framework for adaptive stock scoring, strategy, and selection in portfolio optimization\.External Links:2510\.17393Cited by:[§1](https://arxiv.org/html/2608.07813#S1.p1.1)\.
- I\. Chern, S\. Chern, S\. Chen, W\. Yuan, K\. Feng, C\. Zhou, J\. He, G\. Neubig, and P\. Liu \(2023\)FacTool: factuality detection in generative ai — a tool augmented framework for multi\-task and multi\-domain scenarios\.Note:arXiv:2307\.13528External Links:2307\.13528Cited by:[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px2.p1.1)\.
- K\. Cobbeet al\.\(2021\)Training verifiers to solve math word problems\.Note:arXiv:2110\.14168External Links:2110\.14168Cited by:[§5](https://arxiv.org/html/2608.07813#S5.SS0.SSS0.Px1.p1.5)\.
- J\. Gu, X\. Jiang, Z\. Shi, H\. Tan, X\. Zhai, C\. Xu, W\. Li, Y\. Shen, S\. Ma, H\. Liu, Y\. Wang, and J\. Guo \(2024\)A survey on LLM\-as\-a\-judge\.arXiv preprint arXiv:2411\.15594\.Cited by:[§1](https://arxiv.org/html/2608.07813#S1.p2.1),[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px1.p1.1)\.
- P\. Lewiset al\.\(2020\)Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.33,Red Hook, NY, USA,pp\. 9459–9474\.Cited by:[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px2.p1.1)\.
- H\. Lightmanet al\.\(2023\)Let’s verify step by step\.Note:arXiv:2305\.20050External Links:2305\.20050Cited by:[§1](https://arxiv.org/html/2608.07813#S1.p1.1),[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px2.p1.1)\.
- Q\. Liu, L\. Ye, W\. Ma, Y\. Chou, and A\. Yuille \(2025\)Generative adversarial reasoner: enhancing LLM reasoning with adversarial reinforcement learning\.arXiv preprint arXiv:2512\.16917\.Cited by:[Table 5](https://arxiv.org/html/2608.07813#A1.T5.3.4.3.1),[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px3.p1.1)\.
- Q\. Lu, Z\. Ma, S\. Zhong, J\. Wang, D\. Yu, M\. K\. Ng, and P\. Luo \(2025\)SWIRL: a staged workflow for interleaved reinforcement learning in mobile GUI control\.arXiv preprint arXiv:2508\.20018\.Cited by:[Table 5](https://arxiv.org/html/2608.07813#A1.T5.3.6.5.1),[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px3.p1.1)\.
- S\. Min, K\. Krishna, X\. Lyu, M\. Lewis, W\. Yih, P\. Koh, M\. Iyyer, L\. Zettlemoyer, and H\. Hajishirzi \(2023\)FActScore: fine\-grained atomic evaluation of factual precision in long form text generation\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Singapore,pp\. 12076–12100\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.741)Cited by:[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2608.07813#S4.SS1.p2.1)\.
- R\. Nakano, J\. Hilton, S\. Balaji, J\. Wu, L\. Ouyang, C\. Kim, M\. Nguyen, H\. Jiang, P\. Chen, Y\. Xu,et al\.\(2021\)WebGPT: browser\-assisted question\-answering with human feedback\.Note:arXiv:2112\.09332External Links:2112\.09332Cited by:[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px2.p1.1)\.
- L\. Ouyanget al\.\(2022\)Training language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.35,Red Hook, NY, USA,pp\. 27730–27744\.Cited by:[§1](https://arxiv.org/html/2608.07813#S1.p1.1)\.
- A\. Panickssery, S\. R\. Bowman, and S\. Feng \(2024\)LLM evaluators recognize and favor their own generations\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2404\.13076Cited by:[§1](https://arxiv.org/html/2608.07813#S1.p2.1),[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Santhosh, F\. Abbas, H\. Ahmad, and C\. Szabo \(2026\)Comparative analysis of large language models in healthcare\.External Links:2604\.10316Cited by:[§1](https://arxiv.org/html/2608.07813#S1.p1.1)\.
- Z\. Shaoet al\.\(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.Note:arXiv:2402\.03300External Links:2402\.03300Cited by:[§1](https://arxiv.org/html/2608.07813#S1.p1.1)\.
- B\. Wang, W\. Su, H\. Tian, T\. Yang, Y\. Zhou, T\. Yao, Q\. Ai, and Y\. Liu \(2025\)From <Answer\> to <Think\>: multidimensional supervision of reasoning process for LLM optimization\.arXiv preprint arXiv:2510\.11457\.Cited by:[Table 5](https://arxiv.org/html/2608.07813#A1.T5.3.2.1.1),[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px3.p1.1)\.
- P\. Wang, L\. Li, L\. Chen, Z\. Cai, D\. Zhu, B\. Lin, Y\. Cao, Q\. Liu, T\. Liu, and Z\. Sui \(2024\)Large language models are not fair evaluators\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(ACL\),Note:arXiv:2305\.17926Cited by:[§1](https://arxiv.org/html/2608.07813#S1.p2.1),[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px1.p1.1)\.
- P\. Wanget al\.\(2025\)Math\-shepherd: verify and reinforce llms step\-by\-step without human annotations\.Note:arXiv:2312\.08935External Links:2312\.08935Cited by:[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px2.p1.1)\.
- X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou \(2023\)Self\-consistency improves chain of thought reasoning in language models\.Note:arXiv:2203\.11171External Links:2203\.11171Cited by:[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px2.p1.1),[§3](https://arxiv.org/html/2608.07813#S3.SS0.SSS0.Px2.p1.11)\.
- J\. Wuet al\.\(2025\)Mitigating LLM hallucination via behaviorally calibrated reinforcement learning\.arXiv preprint arXiv:2512\.19920\.Cited by:[Table 5](https://arxiv.org/html/2608.07813#A1.T5.3.3.2.1),[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px3.p1.1)\.
- Z\. Yanget al\.\(2018\)HotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,Brussels, Belgium,pp\. 2369–2380\.External Links:[Document](https://dx.doi.org/10.18653/v1/D18-1259)Cited by:[§5](https://arxiv.org/html/2608.07813#S5.SS0.SSS0.Px1.p1.5)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2023\)ReAct: synergizing reasoning and acting in language models\.Note:ICLR 2023; arXiv:2210\.03629External Links:2210\.03629Cited by:[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Zhang, D\. Goel, H\. Ahmad, and J\. Shen \(2026a\)Beyond reactive agents: uncertainty\-gated meta\-reasoning for tool\-augmented decision\-making\.Note:Available at SSRN 6997675Cited by:[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Zhang, D\. Goel, H\. Ahmad, and J\. Shen \(2026b\)VACS: value\-aligned compositional shielding for multi\-agent reasoning\.Note:Available at SSRN 7115827Cited by:[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px1.p1.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2023\)Judging LLM\-as\-a\-judge with MT\-Bench and chatbot arena\.InAdvances in Neural Information Processing Systems \(NeurIPS\) Datasets and Benchmarks Track,Note:arXiv:2306\.05685Cited by:[§1](https://arxiv.org/html/2608.07813#S1.p1.1),[§2](https://arxiv.org/html/2608.07813#S2.SS0.SSS0.Px1.p1.1)\.

## Appendix APositioning Against Recent Multi\-Signal Methods

Table 5:Positioning against five recent multi\-signal or process\-feedback methods\. “Matched here” means rerun on our frozen candidate pool; dashes mark systems whose published numbers are not protocol\-comparable and are therefore not reproduced as baselines\.
## Appendix BDesign Analysis: Derivations

#### Exact variance decomposition\.

ForR¯=∑i=17wi​Ri\\bar\{R\}=\\sum\_\{i=1\}^\{7\}w\_\{i\}R\_\{i\}with∑iwi=1\\sum\_\{i\}w\_\{i\}=1,wi≥0w\_\{i\}\\geq 0, andRi∈\[0,1\]R\_\{i\}\\in\[0,1\],Var​\(R¯\)=∑iwi2​Var​\(Ri\)\+2​∑i<jwi​wj​Cov​\(Ri,Rj\)\\mathrm\{Var\}\(\\bar\{R\}\)=\\sum\_\{i\}w\_\{i\}^\{2\}\\mathrm\{Var\}\(R\_\{i\}\)\+2\\sum\_\{i<j\}w\_\{i\}w\_\{j\}\\mathrm\{Cov\}\(R\_\{i\},R\_\{j\}\)by bilinearity\. Hence if any pair withwi​wj\>0w\_\{i\}w\_\{j\}\>0hasCov​\(Ri,Rj\)<0\\mathrm\{Cov\}\(R\_\{i\},R\_\{j\}\)<0thenVar​\(R¯\)<∑iwi2​Var​\(Ri\)\\mathrm\{Var\}\(\\bar\{R\}\)<\\sum\_\{i\}w\_\{i\}^\{2\}\\mathrm\{Var\}\(R\_\{i\}\)\. Conflated steps in the sense of Eq\. \([1](https://arxiv.org/html/2608.07813#S3.E1)\) have highRlogicR\_\{\\mathrm\{logic\}\}and lowRfactR\_\{\\mathrm\{fact\}\}by construction, so the anti\-conflation design induces exactly the negative covariance that lowers pre\-gate variance\. If additionally the log\-policy score is uniformlyGG\-Lipschitz, thengt=ϕt​R¯​\(σt\)g\_\{t\}=\\phi\_\{t\}\\bar\{R\}\(\\sigma\_\{t\}\)satisfies𝔼​‖gt‖2≤G2​𝔼​\[R¯2\]\\mathbb\{E\}\\\|g\_\{t\}\\\|^\{2\}\\leq G^\{2\}\\,\\mathbb\{E\}\[\\bar\{R\}^\{2\}\], so reducingVar​\(R¯\)\\mathrm\{Var\}\(\\bar\{R\}\)tightens one factor of the score\-function second\-moment bound\. This is a bound on an estimator, not a learning\-stability guarantee, and it does not apply to the gated rewardRstepR\_\{\\mathrm\{step\}\}, which is nonlinear\.

#### Unsupported\-step rate\.

WithUt=𝟏​\[maxd∈𝒟t⁡fNLI​\(st∣d\)≤η\]U\_\{t\}=\\mathbf\{1\}\[\\max\_\{d\\in\\mathcal\{D\}\_\{t\}\}f\_\{\\mathrm\{NLI\}\}\(s\_\{t\}\\mid d\)\\leq\\eta\], the factual\-grounding channel is a monotone transform of the same statistic,Rfact=\(1\+maxd⁡fNLI\)/2R\_\{\\mathrm\{fact\}\}=\(1\+\\max\_\{d\}f\_\{\\mathrm\{NLI\}\}\)/2, so penalizingRfactR\_\{\\mathrm\{fact\}\}directly penalizes𝔼​\[Ut\]\\mathbb\{E\}\[U\_\{t\}\]conditional on the corpus\. The bound is corpus\-relative: it says nothing about steps whose support is absent from𝒟t\\mathcal\{D\}\_\{t\}\.

#### Calibration channel\.

Repist=1−\|ct−et\|R\_\{\\mathrm\{epist\}\}=1\-\|c\_\{t\}\-e\_\{t\}\|is an absolute\-deviation loss and therefore elicits the median of the model’s evidence\-strength belief rather than its full predictive distribution\. A proper scoring rule would elicit more, at the cost of a channel that is no longer bounded in\[0,1\]\[0,1\]by construction\.

## Appendix CReproducibility

Code, per\-question predictions for all four selectors on both datasets, verifier prompts, timing logs, and the bootstrap/McNemar scripts are released with the paper\. All reported runs use seed 42 for training and seed 2029 for the confirmation split; the frozen candidate pool is distributed so that the selector comparison can be rerun without GPU access\.