Predicting Human-Gain Curves under Local Proxy Optimization from Repeated Ratings

# Predicting Human-Gain Curves under Local Proxy Optimization from Repeated Ratings
Source: [https://arxiv.org/html/2610.41114](https://arxiv.org/html/2610.41114)
Moonwon Choi††thanks:Equal contribution\.Affiliation:Graduate School of Data Science, Seoul National UniversityEmail:[yellowbill@snu\.ac\.kr](mailto:)Kisung Nam11footnotemark:1Affiliation:Massachusetts General Hospital, Harvard Medical SchoolAffiliation:Broad Institute of MIT and HarvardEmail:[knam@broadinstitute\.org](mailto:)Seunggeun Lee††thanks:Corresponding author:lee7801@snu\.ac\.kr\.Affiliation:Graduate School of Data Science, Seoul National UniversityEmail:[lee7801@snu\.ac\.krhttps://github\.com/pacifiic/local\-proxy\-diagnostics](mailto:[email protected]://github.com/pacifiic/local-proxy-diagnostics)

###### Abstract

Optimizing a proxy objective can improve human\-perceived quality, but the resulting gain can vary with optimization strength\. Average proxy–human correlation alone does not reveal how human\-perceived quality changes as optimization strength increases\. We proposePACE\(Proxy Alignment and Curvature Expansion\), a pre\-optimization diagnostic that uses repeated human ratings to predict the human\-gain curve induced by a fixed proxy\. The method models proxy optimization as a KL\-regularized exponential tilt\. At KL radiusdd, the local gain is approximated byA​d\+B​dA\\sqrt\{d\}\+Bd, whereAAmeasures first\-order proxy–human alignment andBBcaptures the leading second\-order correction along the proxy\-optimization path\. We estimateAAandBBfrom repeated human ratings on calibration items and test the resulting gain\-curve prediction on held\-out evaluation items\. Across SummEval, analysis\-eligible WMT MQM clusters, an OpenMEVA\-MAGS ROC/WP repeated\-rating slice, and a GPT\-4o judge proxy experiment, PACE consistently reduces held\-out gain\-prediction error relative toAA\-only, zero\-gain, and reliability\-only baselines\. With the same pre\-pool information, it is competitive with flexible proxy\-conditional calibration, and a LoRA\-DPO experiment provides a modest bridge to actual optimization\. Transfer experiments across datasets and evaluation slices show that source coefficients are useful priors, but target calibration further improves prediction\. The resulting diagnostic is local and one\-sided: it characterizes gentle optimization along the positive tilt branch of a fixed item pool\.

## 1Introduction

Post\-training and inference\-time selection often improve model outputs by optimizing a measurable proxy for human preference\. Reinforcement Learning from Human Feedback \(RLHF\) learns reward models from human comparisons and trains policies against the learned rewards\[[7](https://arxiv.org/html/2610.41114#bib.bib7),[43](https://arxiv.org/html/2610.41114#bib.bib43),[36](https://arxiv.org/html/2610.41114#bib.bib36),[2](https://arxiv.org/html/2610.41114#bib.bib2)\]\. Direct alignment methods optimize preference\-derived objectives without explicitly fitting a reward model\[[39](https://arxiv.org/html/2610.41114#bib.bib39)\]\. Best\-of\-NN, reranking, and judge\-based selection apply analogous proxy pressure at inference time by selecting outputs with high reward or judge scores\[[43](https://arxiv.org/html/2610.41114#bib.bib43),[4](https://arxiv.org/html/2610.41114#bib.bib4)\]\. Although these methods can be effective, the human benefit of proxy optimization is not fixed by average proxy–human correlation: as optimization strength increases, human\-perceived quality may improve, saturate, or eventually decline\[[43](https://arxiv.org/html/2610.41114#bib.bib43),[15](https://arxiv.org/html/2610.41114#bib.bib15),[37](https://arxiv.org/html/2610.41114#bib.bib37),[26](https://arxiv.org/html/2610.41114#bib.bib26),[32](https://arxiv.org/html/2610.41114#bib.bib32),[42](https://arxiv.org/html/2610.41114#bib.bib42)\]\. Thus, the practical question is not only whether a proxy is correlated with human judgments on average, but how much human gain its local optimization is expected to induce as optimization strength varies\.

Repeated human ratings make this question testable\. Proxy optimization upweights high\-proxy items, so the relevant target is the human quality of the locally reweighted high\-proxy region, not annotator agreement or global proxy–human association alone\. Repeated ratings give multiple noisy views of the same item\-level quality\. In direct human\-proxy settings, one view can serve as the proxy and the remaining ratings can define a leave\-one\-rater\-out human gold target\. For external proxies, such as learned rewards or large\-language\-model \(LLM\) judges, the same repeated\-rating matrix supplies the human calibration and evaluation target\. Disjoint items can then be reserved for prospective evaluation\. Reliability tells us whether the human target is observable, but the magnitude of proxy\-induced human gain depends on proxy–gold alignment and the quality of the high\-proxy region, not on measurement stability alone\. Our method, PACE \(Proxy Alignment and Curvature Expansion\), is predictive: a small repeated\-rating calibration matrix can estimate two local coefficients, the PACE coefficients, that forecast the held\-out human\-gain curve induced by a fixed proxy\. For example, suppose that 300 items with repeated ratings are available now, and the decision is whether to generate a pool of 1,000 candidates and optimize it against a reward model\. The coefficients are estimated from the 300 rated items before the larger pool exists\. The forecast then states how much human\-rated quality a given optimization strength should buy within a checked range, or signals that more ratings, gentler selection, or abstention is warranted \(Figure[1](https://arxiv.org/html/2610.41114#S1.F1)\)\.

Figure 1:Using PACE\. \(a\) Proxy scoresrrandk≥2k\\geq 2human ratingsY1,…,YkY\_\{1\},\\dots,Y\_\{k\}on calibration items give plug\-in estimates of the alignment coefficientAAand the curvature coefficientBB, which forecast the local human\-gain curve before optimization\. A planned rule that is not a raw\-score tilt, such as Best\-of\-NN, is first translated into its induced KL radius and relative effective sample size \(RelESS\) on an unlabeled, proxy\-scored pilot \(dashed\)\. No human labels of the target pool are used\. \(b\) Schematic withB^<0\\hat\{B\}<0:A^​d\\hat\{A\}\\sqrt\{d\}alone overstates the gain, andB^​d\\hat\{B\}dcorrects it inside the checked window; beyond it the realized gain may saturate or decline, and no forecast is made\.We study these coefficients on a smooth finite\-item proxy\-optimization path\. Given a reference item distributionπref\\pi\_\{\\mathrm\{ref\}\}and proxy scorerr, we model small proxy pressure by the exponential reweightingπt​\(i\)∝πref​\(i\)​exp⁡\(t​ri\)\\pi\_\{t\}\(i\)\\propto\\pi\_\{\\mathrm\{ref\}\}\(i\)\\exp\(tr\_\{i\}\)\. Increasingttgradually concentrates mass on high\-proxy items\. The corresponding Kullback–Leibler \(KL\) radiusd=KL\(πt∥πref\)d=\\operatorname\{KL\}\(\\pi\_\{t\}\\\|\\pi\_\{\\mathrm\{ref\}\}\)measures the strength of this intervention\.

LetJJdenote human gold quality, and letQ⁡\(d\)Q\(d\)be expected human quality along this path\. The local gain curve then satisfies

Q⁡\(d\)=Q⁡\(0\)\+A​d\+B​d\+O⁡\(d3/2\)\.Q\(d\)=Q\(0\)\+A\\sqrt\{d\}\+Bd\+O\(d^\{3/2\}\)\.\(1\)The alignment coefficientAAmeasures standardized proxy–gold alignment in the direction in which optimization moves the distribution\. The curvature coefficientBBis the quadratic term inx=dx=\\sqrt\{d\}, soA\>0A\>0andB<0B<0mean that the gain begins to saturate\. Their signs and magnitudes are properties of the feedback geometry, that is, the joint structure ofJJandrrestimated from calibration data\. Eq\. \([1](https://arxiv.org/html/2610.41114#S1.E1)\) is local and one\-sided: it describes small KL radii on the positive tilt branch of a fixed item pool, not the terminal regime of strong optimization, and a negativeBBis leading curvature rather than a predicted collapse\. Discrete selection rules such as Best\-of\-NNand actual policy training therefore enter as bridges, after translation into an induced KL radius, not as targets of the theorem\.

We evaluate PACE prospectively: coefficients estimated on calibration items predict gain curves on disjoint evaluation items \(Section[4](https://arxiv.org/html/2610.41114#S4)\)\. Across SummEval, eligible WMT Multidimensional Quality Metrics \(MQM\) clusters, OpenMEVA\-MAGS, and a GPT\-4o external\-judge proxy, the PACE forecast has lower held\-out error than anAA\-only predictor, a zero\-gain predictor, and a reliability\-only surrogate\. Flexible proxy\-conditional calibration by binning, splines, or isotonic regression is the natural alternative\. When every method sees only the calibration items, the PACE forecast is competitive with these calibrators, and when the proxy scores of the target pool are also available, the two\-term projection of each fitted calibrator closely tracks its exact integration\. A three\-seed LoRA\-DPO experiment scored by a frozen silver reward model gives a modest bridge to actual optimization, in which the realized gain is about half of the idealized tilt gain at the same radius\. Cross\-slice transfer shows that source coefficients are useful priors that target calibration improves\.

Contributions\.We make three contributions\. First, we define the human\-gain curve induced by KL\-regularized proxy optimization on a finite item set\. Second, we derive a scale\-free two\-coefficient local law,A​d\+B​dA\\sqrt\{d\}\+Bd, in whichAAmeasures standardized proxy–human alignment andBBthe leading curvature\. Third, we turn repeated human\-rating matrices into a prospective calibration protocol for held\-out human\-gain curves, compare it with flexible calibration under matched information, and check discrete selection rules and actual DPO training against the checked KL window\.

## 2Related work

Proxy optimization, KL control, and gain prediction\.Preference\-based alignment methods learn from human feedback through explicit reward modeling\[[7](https://arxiv.org/html/2610.41114#bib.bib7),[43](https://arxiv.org/html/2610.41114#bib.bib43),[36](https://arxiv.org/html/2610.41114#bib.bib36),[2](https://arxiv.org/html/2610.41114#bib.bib2)\]or direct preference objectives\[[39](https://arxiv.org/html/2610.41114#bib.bib39)\]\. KL\-regularized reward optimization is central to RLHF and controlled generation\[[1](https://arxiv.org/html/2610.41114#bib.bib1)\]: early RLHF systems penalize deviation from a reference policy\[[48](https://arxiv.org/html/2610.41114#bib.bib48)\], and KL\-regularized RL can be interpreted through variational inference\[[25](https://arxiv.org/html/2610.41114#bib.bib25)\]\. In the nonparametric fixed\-proxy limit, this objective yields an exponential tilt of the reference distribution, which is the analytic frontier used below\. A related line of work studies reward\-model overoptimization, objective mismatch, direct\-alignment overoptimization, Goodhart effects, and reward hacking\[[15](https://arxiv.org/html/2610.41114#bib.bib15),[37](https://arxiv.org/html/2610.41114#bib.bib37),[26](https://arxiv.org/html/2610.41114#bib.bib26),[40](https://arxiv.org/html/2610.41114#bib.bib40),[16](https://arxiv.org/html/2610.41114#bib.bib16),[32](https://arxiv.org/html/2610.41114#bib.bib32),[42](https://arxiv.org/html/2610.41114#bib.bib42)\]\. These works show that optimizing a proxy score does not by itself determine improvement in the intended human objective\. We ask a complementary pre\-optimization question: what human\-gain curve will a fixed proxy induce under small\-KL optimization on the item set to be evaluated? A direct alternative to an analytic law is to estimate the proxy\-conditional mean𝔼⁡\[J∣r\]\\mathbb\{E\}\[J\\mid r\]by binning, isotonic regression, or penalized splines\[[3](https://arxiv.org/html/2610.41114#bib.bib3),[11](https://arxiv.org/html/2610.41114#bib.bib11)\]and integrate it along the tilt\. The relevant contrast is not between opaque and transparent algorithms but between what each method estimates and what information it can see, so we compare the two under matched information \(Section[5\.1](https://arxiv.org/html/2610.41114#S5.SS1)\)\.

Inference\-time selection, reward\-model evaluation, and LLM judges\.Best\-of\-NN, reranking, and judge\-based selection optimize scalar proxy scores at inference time, and recent work analyzes best\-of\-NN\-style policies theoretically and algorithmically\[[22](https://arxiv.org/html/2610.41114#bib.bib22),[4](https://arxiv.org/html/2610.41114#bib.bib4)\]\. Reward\-model benchmarks evaluate whether learned rewards rank preference pairs correctly across challenging model outputs and domains\[[27](https://arxiv.org/html/2610.41114#bib.bib27)\]\. LLM\-as\-a\-judge methods provide scalable evaluators\[[47](https://arxiv.org/html/2610.41114#bib.bib47)\], but their reliability depends on prompt, model, comparison format, and evaluation design\[[29](https://arxiv.org/html/2610.41114#bib.bib29),[45](https://arxiv.org/html/2610.41114#bib.bib45),[21](https://arxiv.org/html/2610.41114#bib.bib21),[46](https://arxiv.org/html/2610.41114#bib.bib46)\]\. PACE asks a different question: how much held\-out human gain is induced by locally optimizing such a proxy\. Best\-of\-NNis used as a bridge to discrete inference\-time selection, while the GPT\-4o judge experiment tests PACE with an external LLM\-based proxy rather than making benchmark claims about GPT\-4o as a general evaluator\.

Repeated human evaluation and reliability\.Human\-evaluation datasets show substantial annotator and protocol variation, motivating reliability analysis in summarization, machine translation, and broader NLP evaluation\[[12](https://arxiv.org/html/2610.41114#bib.bib12),[13](https://arxiv.org/html/2610.41114#bib.bib13),[5](https://arxiv.org/html/2610.41114#bib.bib5),[18](https://arxiv.org/html/2610.41114#bib.bib18)\]\. Classical intraclass\-correlation methods quantify the item\-level component of repeated ratings\[[41](https://arxiv.org/html/2610.41114#bib.bib41),[33](https://arxiv.org/html/2610.41114#bib.bib33)\]\. This literature asks how stable or reproducible human annotations are\. We use repeated ratings for a different purpose: predicting the held\-out human\-gain curve induced by optimizing a scalar proxy\. Reliability anchors the observation model, while the coefficients below model the intervention outcome\.

## 3Local diagnostic

Observation process\.LetJiJ\_\{i\}be the gold quality of itemiiunder a reference distributionπref\\pi\_\{\\mathrm\{ref\}\}, and letrir\_\{i\}be a proxy score\. In experiments,JJis a held\-out human aggregate, not an unobserved latent truth\. In a simple repeated\-rating modelYi​j=Ji\+ηi​jY\_\{ij\}=J\_\{i\}\+\\eta\_\{ij\}, with ratersjj, one rating view can serve asrrand the remaining views define a leave\-one\-rater\-out gold target; for external proxies such as learned rewards or LLM judges, the repeated ratings supply the human target\. Intraclass\-correlation \(ICC\) reliability, the share of rating variance due to the item term\[[41](https://arxiv.org/html/2610.41114#bib.bib41),[33](https://arxiv.org/html/2610.41114#bib.bib33)\], anchors this measurement model, but the gain predictor below depends on proxy–gold alignment and curvature, not on reliability alone\.

Tilt path and KL\-regularized frontier\.For a fixed proxy scorerron the reference distributionπref\\pi\_\{\\mathrm\{ref\}\}, we model local proxy optimization by the exponential\-tilt path

d​πtd​πref​\(i\)=exp⁡\(t​ri\)𝔼πref​exp⁡\(t​r\),t≥0\.\\frac\{d\\pi\_\{t\}\}\{d\\pi\_\{\\mathrm\{ref\}\}\}\(i\)=\\frac\{\\exp\(tr\_\{i\}\)\}\{\\mathbb\{E\}\_\{\\pi\_\{\\mathrm\{ref\}\}\}\\exp\(tr\)\},\\qquad t\\geq 0\.\(2\)Increasingttreweights the reference distribution toward high\-proxy items\. We measure the strength of this intervention by the induced KL radiusd\(t\)=KL\(πt∥πref\)d\(t\)=\\operatorname\{KL\}\(\\pi\_\{t\}\\\|\\pi\_\{\\mathrm\{ref\}\}\), and writeQ​\(t\)=𝔼πt​\[J\]Q\(t\)=\\mathbb\{E\}\_\{\\pi\_\{t\}\}\[J\]for the corresponding expected human gold quality\. Wheneverd⁡\(t\)d\(t\)is locally invertible on the positive branch, we writeQ⁡\(d\)=Q⁡\(t⁡\(d\)\)Q\(d\)=Q\(t\(d\)\)and callΔ⁡\(d\)=Q⁡\(d\)−Q⁡\(0\)\\Delta\(d\)=Q\(d\)\-Q\(0\)the human\-gain curve induced by the fixed proxy\.

The tilt path in Eq\. \([2](https://arxiv.org/html/2610.41114#S3.E2)\) is the nonparametric solution of KL\-regularized proxy maximization, a classicalII\-projection / variational identity\[[8](https://arxiv.org/html/2610.41114#bib.bib8),[10](https://arxiv.org/html/2610.41114#bib.bib10)\]\. The proof is in Appendix[A\.1](https://arxiv.org/html/2610.41114#A1.SS1)\.

###### Proposition 1\(Exponential tilt as the KL\-regularized frontier\)\.

Fixt\>0t\>0and assume𝔼πref​\[\(1\+\|r\|\)​et​r\]<∞\\mathbb\{E\}\_\{\\pi\_\{\\mathrm\{ref\}\}\}\[\(1\+\|r\|\)e^\{tr\}\]<\\infty\. Then, overπ\\piwith finite KL,

supπ≪πref\{t𝔼π\[r\]−KL\(π∥πref\)\}=log𝔼πrefet​r,\\sup\_\{\\pi\\ll\\pi\_\{\\mathrm\{ref\}\}\}\\left\\\{t\\,\\mathbb\{E\}\_\{\\pi\}\[r\]\-\\operatorname\{KL\}\(\\pi\\\|\\pi\_\{\\mathrm\{ref\}\}\)\\right\\\}=\\log\\mathbb\{E\}\_\{\\pi\_\{\\mathrm\{ref\}\}\}e^\{tr\},\(3\)with unique maximizerπt\\pi\_\{t\}of Eq\. \([2](https://arxiv.org/html/2610.41114#S3.E2)\), which therefore maximizes𝔼π\[r\]−t−1KL\(π∥πref\)\\mathbb\{E\}\_\{\\pi\}\[r\]\-t^\{\-1\}\\operatorname\{KL\}\(\\pi\\\|\\pi\_\{\\mathrm\{ref\}\}\)\. On any regular interval whered⁡\(t\)d\(t\)is strictly increasing, the sameπt\\pi\_\{t\}solves the corresponding fixed\-KL problem\. The terminal regime, whereπt\\pi\_\{t\}concentrates on proxy maximizers, is outside our scope\.

Scale\-free coefficient law\.The following coefficient law is scale\-free: the proxy score need not be on the same numerical scale as human gold\. All moments and cumulants below are taken underπref\\pi\_\{\\mathrm\{ref\}\}unless otherwise stated\. Letσr2=Varπref⁡\(r\)\\sigma\_\{r\}^\{2\}=\\operatorname\{Var\}\_\{\\pi\_\{\\mathrm\{ref\}\}\}\(r\),κr=κ3​\(r,r,r\)\\kappa\_\{r\}=\\kappa\_\{3\}\(r,r,r\), andκJ​r​r=κ3​\(J,r,r\)\\kappa\_\{Jrr\}=\\kappa\_\{3\}\(J,r,r\)\.

###### Theorem 1\(Local coefficient law\)\.

Assumeσr\>0\\sigma\_\{r\}\>0and eitherJ,rJ,rare bounded or the joint cumulant\-generating function of\(J,r\)\(J,r\)isC4C^\{4\}in a neighborhood of the origin\. Along the positive exponential\-tilt branch in Eq\. \([2](https://arxiv.org/html/2610.41114#S3.E2)\), asd↓0d\\downarrow 0,

Q⁡\(d\)=Q⁡\(0\)\+A​d\+B​d\+O⁡\(d3/2\),Q\(d\)=Q\(0\)\+A\\sqrt\{d\}\+Bd\+O\(d^\{3/2\}\),\(4\)where

A=2​Cov⁡\(J,r\)σr,B=κJ​r​rσr2−2​Cov⁡\(J,r\)​κr3​σr4\.A=\\sqrt\{2\}\\frac\{\\operatorname\{Cov\}\(J,r\)\}\{\\sigma\_\{r\}\},\\qquad B=\\frac\{\\kappa\_\{Jrr\}\}\{\\sigma\_\{r\}^\{2\}\}\-\\frac\{2\\operatorname\{Cov\}\(J,r\)\\kappa\_\{r\}\}\{3\\sigma\_\{r\}^\{4\}\}\.\(5\)

Standardized interpretation\.LetZ=\(r−𝔼​r\)/σrZ=\(r\-\\mathbb\{E\}r\)/\\sigma\_\{r\}andJ¯=J−𝔼​J\\bar\{J\}=J\-\\mathbb\{E\}J\. Then the same coefficients can be written as

A=2​𝔼​\[J¯​Z\],B=𝔼⁡\[J¯​Z2\]⏟second\-order proxy–gold moment−23​𝔼​\[J¯​Z\]​𝔼​\[Z3\]⏟proxy\-skewness correction\.A=\\sqrt\{2\}\\,\\mathbb\{E\}\[\\bar\{J\}Z\],\\qquad B=\\underbrace\{\\mathbb\{E\}\[\\bar\{J\}Z^\{2\}\]\}\_\{\\text\{second\-order proxy\-\-gold moment\}\}\-\\underbrace\{\\tfrac\{2\}\{3\}\\mathbb\{E\}\[\\bar\{J\}Z\]\\mathbb\{E\}\[Z^\{3\}\]\}\_\{\\text\{proxy\-skewness correction\}\}\.\(6\)The coefficientAAmeasures first\-order standardized alignment between proxy and gold\. The coefficientBBis the leading curvature term on the positive tilt branch\. Its first term is a two\-sided moment: it is positive when items far from the proxy mean, on either side, have above\-average gold quality\. The skewness term is a coordinate correction that appears because the path is indexed by the KL radius rather than by the standardized tilts=t​σrs=t\\sigma\_\{r\}\. Inss,Q−Q⁡\(0\)=𝔼⁡\[J¯​Z\]​s\+12​𝔼​\[J¯​Z2\]​s2\+O⁡\(s3\)Q\-Q\(0\)=\\mathbb\{E\}\[\\bar\{J\}Z\]s\+\\tfrac\{1\}\{2\}\\mathbb\{E\}\[\\bar\{J\}Z^\{2\}\]s^\{2\}\+O\(s^\{3\}\)andd=12​s2\+13​𝔼​\[Z3\]​s3\+O⁡\(s4\)d=\\tfrac\{1\}\{2\}s^\{2\}\+\\tfrac\{1\}\{3\}\\mathbb\{E\}\[Z^\{3\}\]s^\{3\}\+O\(s^\{4\}\), sos⁡\(d\)=2​d−23​𝔼​\[Z3\]​d\+O⁡\(d3/2\)s\(d\)=\\sqrt\{2d\}\-\\tfrac\{2\}\{3\}\\mathbb\{E\}\[Z^\{3\}\]d\+O\(d^\{3/2\}\)\. With positive alignment, a right\-skewed proxy spends more KL per unit ofss, so a fixed radius buys less tilt and the correction is negative; left skew makes it positive\. The correction vanishes when either alignment or skewness is zero, and it is comparable to the first term when23​\|𝔼⁡\[J¯​Z\]​𝔼​\[Z3\]\|≈\|𝔼⁡\[J¯​Z2\]\|\\tfrac\{2\}\{3\}\|\\mathbb\{E\}\[\\bar\{J\}Z\]\\mathbb\{E\}\[Z^\{3\}\]\|\\approx\|\\mathbb\{E\}\[\\bar\{J\}Z^\{2\}\]\|\(Appendix[A\.4](https://arxiv.org/html/2610.41114#A1.SS4)\)\.

Proxy\-error interpretation\.When proxy and gold share a rating scale, write

r−𝔼​r=\(J−𝔼​J\)\+ε,ρ=Cov⁡\(J,ε\)\.r\-\\mathbb\{E\}r=\(J\-\\mathbb\{E\}J\)\+\\varepsilon,\\qquad\\rho=\\operatorname\{Cov\}\(J,\\varepsilon\)\.\(7\)ThenCov⁡\(J,r\)=σJ2\+ρ\\operatorname\{Cov\}\(J,r\)=\\sigma\_\{J\}^\{2\}\+\\rho, so reliability alone can be miscalibrated when proxy error is correlated with gold quality\. Judges, learned rewards, and rerankers need not share this scale\.

Why correlation is not enough\.A global proxy–gold correlation compresses the joint distribution into one first\-order association\. Proxy optimization instead reweights the item set toward high proxy scores, andBBcaptures the leading correction as the tilt begins to concentrate there\. Two proxies can therefore have similar reliability or correlation but different local gain curves\.

Scope of the local readout\.The coefficient readout is local and one\-sided\. Both coefficients describe a specified finite item set, a fixed proxy, and the positive tilt direction near the reference distribution\. A positiveAAsupports optimization only inside a checked neighborhood, and, withA\>0A\>0, a negativeBBsignals leading saturation, not an eventual downturn or a universal collapse law\. The law says nothing about the terminal regime, the negative branch, or rules that are not raw\-score tilts\. The empirical protocol therefore pairs coefficient prediction with direct finite\-range diagnostics: if tilted variance collapses, the effective sample size \(ESS\) becomes small, or the target radius is unreachable, the checked KL window should be narrowed\. Appendix[A\.10](https://arxiv.org/html/2610.41114#A1.SS10)summarizes coefficient sign patterns and their local implications\.

Finite\-range structure\.Theorem[1](https://arxiv.org/html/2610.41114#Thmtheorem1)is local, so finite\-range use requires diagnostics rather than extrapolation\. Two exact identities motivate them\. First,d′​\(t\)=t​Varπt⁡\(r\)d^\{\\prime\}\(t\)=t\\operatorname\{Var\}\_\{\\pi\_\{t\}\}\(r\), so if0<v−≤Varπt⁡\(r\)≤v\+0<v\_\{\-\}\\leq\\operatorname\{Var\}\_\{\\pi\_\{t\}\}\(r\)\\leq v\_\{\+\}on\[0,T\]\[0,T\], then

v−2​t2≤d⁡\(t\)≤v\+2​t2,\\tfrac\{v\_\{\-\}\}\{2\}t^\{2\}\\leq d\(t\)\\leq\\tfrac\{v\_\{\+\}\}\{2\}t^\{2\},\(8\)andd\\sqrt\{d\}is locally proportional to the tilt strengthtt, which is why Eq\. \([4](https://arxiv.org/html/2610.41114#S3.E4)\) is written in powers ofd\\sqrt\{d\}\. Second, ifh⁡\(u\)=𝔼πref​\[J∣r=u\]h\(u\)=\\mathbb\{E\}\_\{\\pi\_\{\\mathrm\{ref\}\}\}\[J\\mid r=u\], then

d​Qd​d​\(d⁡\(t\)\)=Covπt⁡\(J,r\)t​Varπt​\(r\)=Covπt⁡\(h⁡\(r\),r\)t​Varπt​\(r\)\.\\frac\{dQ\}\{dd\}\(d\(t\)\)=\\frac\{\\operatorname\{Cov\}\_\{\\pi\_\{t\}\}\(J,r\)\}\{t\\operatorname\{Var\}\_\{\\pi\_\{t\}\}\(r\)\}=\\frac\{\\operatorname\{Cov\}\_\{\\pi\_\{t\}\}\(h\(r\),r\)\}\{t\\operatorname\{Var\}\_\{\\pi\_\{t\}\}\(r\)\}\.\(9\)This expresses the gain slope through the proxy\-conditional gold structure, the route to the finite\-range residual bound in Appendix[A\.5](https://arxiv.org/html/2610.41114#A1.SS5)\. The proof of Theorem[1](https://arxiv.org/html/2610.41114#Thmtheorem1)is in Appendix[A\.2](https://arxiv.org/html/2610.41114#A1.SS2)\.

## 4Estimation protocol

Finite\-item target\.For a finite item set𝒟\\mathcal\{D\}, the reference distribution is uniform andπt​\(i\)∝exp⁡\(t​ri\)\\pi\_\{t\}\(i\)\\propto\\exp\(tr\_\{i\}\)\. For each proxy–gold path, we trace the exact finite\-item target curve

Q⁡\(t\)=∑iπt​\(i\)​Ji,d⁡\(t\)=∑iπt​\(i\)​log⁡\(\|𝒟\|​πt​\(i\)\),Q\(t\)=\\sum\_\{i\}\\pi\_\{t\}\(i\)J\_\{i\},\\qquad d\(t\)=\\sum\_\{i\}\\pi\_\{t\}\(i\)\\log\(\|\\mathcal\{D\}\|\\pi\_\{t\}\(i\)\),and score predictions of the gainΔ⁡\(d\)\\Delta\(d\)over the checked windowd≤0\.10d\\leq 0\.10\. Calibration moments give the plug\-in PACE forecast

Δ^​\(d\)=A^​d\+B^​d,\\widehat\{\\Delta\}\(d\)=\\hat\{A\}\\sqrt\{d\}\+\\hat\{B\}d,\(10\)labeledA\+BA\{\+\}Bbelow, where hatted coefficients are calibration estimates \(Appendix[A\.6](https://arxiv.org/html/2610.41114#A1.SS6)\)\. Human labels of evaluation items are used only to compute the target curve and scores\. In human\-proxy paths, one rater supplies the proxyrir\_\{i\}and the remaining raters define leave\-one\-rater\-out \(LOO\) goldJiJ\_\{i\}\. In external\-proxy paths, a model suppliesrir\_\{i\}and repeated human ratings defineJiJ\_\{i\}\. The main experiments use a pooled tilt over the item set, which may change the prompt marginal; Appendix[A\.7](https://arxiv.org/html/2610.41114#A1.SS7)discusses prompt\-conditional selection\. Gains and errors are in the native units of each human target \(calibration\-standardized for OOD transfer\) and are compared within, not across, datasets\.

Prospective calibration protocol\.Each repeated\-rating path is split by items into a 30% calibration set and a disjoint 70% evaluation set, and the split is resampled 50 times per path\. Calibration and evaluation items may share prompts and systems, so this tests held\-out\-item prediction within a slice rather than unseen\-prompt or unseen\-system generalization\. Repeats reuse the same path, so intervals are clustered by path\. The slices are SummEval \(32 held\-out\-rater paths\), eligible WMT MQM clusters \(141 paths from 47 complete rater\-set clusters\), OpenMEVA\-MAGS ROC/WP \(10 paths\), and a GPT\-4o judge proxy\[[34](https://arxiv.org/html/2610.41114#bib.bib34)\]with SummEval human targets \(4 paths\)\. They keep only systems, raters, and dimensions supporting finite\-item paths and disjoint gold \(Table[B\.2](https://arxiv.org/html/2610.41114#A2.T2)\)\.

Baselines\.Besides theAA\-only predictorA^​d\\hat\{A\}\\sqrt\{d\}and zero gain, the reliability\-only surrogateAν​dA\_\{\\nu\}\\sqrt\{d\}withAν=2​σ^J2/σ^rA\_\{\\nu\}=\\sqrt\{2\}\\,\\hat\{\\sigma\}\_\{J\}^\{2\}/\\hat\{\\sigma\}\_\{r\}uses only calibration variances of the human aggregate and the proxy\. It equalsAAwhenρ=0\\rho=0in Eq\. \([7](https://arxiv.org/html/2610.41114#S3.E7)\) but otherwise ignores proxy–gold alignment \(Appendix[A\.9](https://arxiv.org/html/2610.41114#A1.SS9)and Appendix[B\.5](https://arxiv.org/html/2610.41114#A2.SS5)\)\. Two data\-driven baselines use the exact calibration\-pool tilt curve: the*calibration two\-term fit*fitsa​d\+b​da\\sqrt\{d\}\+bdto it by least squares, and the*stored empirical curve*transfers it\.

Information regimes and flexible calibration\.A natural alternative estimates the proxy\-conditional meanh⁡\(r\)≈𝔼⁡\[J∣r\]h\(r\)\\approx\\mathbb\{E\}\[J\\mid r\]on calibration items and integrates it along the tilt\. We use equal\-frequency*binned*means \(piecewise constant, not monotone\), a penalized cubic*spline*\(smooth\), and*isotonic*regression \(nondecreasing\), with the bin count and spline penalty chosen by calibration\-internal cross\-validation \(Appendix[B\.7](https://arxiv.org/html/2610.41114#A2.SS7)\)\. The comparison depends on what is visible when the forecast is made\. In the*pre\-pool*regime, which matches the decision whether to build a larger pool, every method sees only the rated calibration items, and flexible calibrators integrateh^\\hat\{h\}over the calibration\-pool tilt\. In the*target\-pool\-aware*regime, the proxy scores of the target pool are also available, but not its human labels\. The same fittedh^\\hat\{h\}is then used in two ways\.*Direct integration*evaluates∑i\{πt⁡\(d\)​\(i\)−π0​\(i\)\}​h^​\(ri\)\\sum\_\{i\}\\\{\\pi\_\{t\(d\)\}\(i\)\-\\pi\_\{0\}\(i\)\\\}\\hat\{h\}\(r\_\{i\}\)on the exact target\-pool tilt\.*Projection*inserts the pseudo\-targeth^​\(r\)\\hat\{h\}\(r\)into the target\-pool moment formulas and predictsAh​d\+Bh​dA\_\{h\}\\sqrt\{d\}\+B\_\{h\}d\. Their difference isolates the loss from truncating a calibrated curve to two terms\. Target labels only score forecasts\.

Evaluation metrics\.Endpoint MAE atd=0\.10d=0\.10, pooled over valid splits, summarizes prediction at the largest checked radius\. CurveMAE averages absolute error overd∈\{0\.01,0\.025,0\.05,0\.075,0\.10\}d\\in\\\{0\.01,0\.025,0\.05,0\.075,0\.10\\\}, first within each path over the \(split, radius\) cells that every compared method can predict and then equally over paths; paired differences use 2,000\-draw path\-clustered bootstrap intervals\. Curvature\-sign agreement, a secondary and margin\-sensitive diagnostic, comparessign⁡\(B^cal\)\\operatorname\{sign\}\(\\hat\{B\}\_\{\\rm cal\}\)with the sign of a coefficient fitted to the exact evaluation curve over0<d≤0\.100<d\\leq 0\.10, not with the moment coefficientBevalB\_\{\\rm eval\}below\.

Prediction\-error decomposition\.Fix a local KL radiusD\>0D\>0and suppose the evaluation population satisfiesΔeval​\(d\)=Aeval​d\+Beval​d\+Reval​\(d\)\\Delta\_\{\\rm eval\}\(d\)=A\_\{\\rm eval\}\\sqrt\{d\}\+B\_\{\\rm eval\}d\+R\_\{\\rm eval\}\(d\)with\|Reval​\(d\)\|≤Ceval​d3/2\|R\_\{\\rm eval\}\(d\)\|\\leq C\_\{\\rm eval\}d^\{3/2\}for alld∈\[0,D\]d\\in\[0,D\]\.

###### Lemma 1\(Calibration prediction\-error decomposition\)\.

For alld∈\[0,D\]d\\in\[0,D\],

\|Δ^​\(d\)−Δeval​\(d\)\|≤d​\|A^−Aeval\|\+d​\|B^−Beval\|\+Ceval​d3/2\.\|\\widehat\{\\Delta\}\(d\)\-\\Delta\_\{\\rm eval\}\(d\)\|\\leq\\sqrt\{d\}\\,\|\\hat\{A\}\-A\_\{\\rm eval\}\|\+d\\,\|\\hat\{B\}\-B\_\{\\rm eval\}\|\+C\_\{\\rm eval\}d^\{3/2\}\.\(11\)Moreover,\|A^−Aeval\|≤\|A^−Acal\|\+\|Acal−Aeval\|\|\\hat\{A\}\-A\_\{\\rm eval\}\|\\leq\|\\hat\{A\}\-A\_\{\\rm cal\}\|\+\|A\_\{\\rm cal\}\-A\_\{\\rm eval\}\|, and analogously forBB\.

The bound separates calibration estimation error, calibration–evaluation shift, and the local remainder\. Under the bounded i\.i\.d\. model of Appendix[A\.6](https://arxiv.org/html/2610.41114#A1.SS6),\|A^−Acal\|,\|B^−Bcal\|=Op\(ncal−1/2\)\|\\hat\{A\}\-A\_\{\\rm cal\}\|,\|\\hat\{B\}\-B\_\{\\rm cal\}\|=O\_\{p\}\(n\_\{\\rm cal\}^\{\-1/2\}\), so shift and the finite\-range remainder are not removed by more calibration data\.

Selection rules and the checked window\.A planned rule that is not a raw\-score tilt, such as Best\-of\-NN\(BoN\), is first translated into the distributionppit induces on the reference poolp0p\_\{0\}\. For BoN,pip\_\{i\}is the exact probability that itemiiwins amongNNreference draws under uniform tie\-breaking, which is computable from unlabeled proxy scores \(Appendix[A\.7](https://arxiv.org/html/2610.41114#A1.SS7)\)\. We report

d=KL\(p∥p0\),RelESS\(p,p0\)=\(∑ipi2/p0,i\)−1,d=\\operatorname\{KL\}\(p\\\|p\_\{0\}\),\\qquad\\mathrm\{RelESS\}\(p,p\_\{0\}\)=\\Big\(\\sum\\nolimits\_\{i\}p\_\{i\}^\{2\}/p\_\{0,i\}\\Big\)^\{\-1\},\(12\)the induced radius and the effective support that remains relative top0p\_\{0\}\[[23](https://arxiv.org/html/2610.41114#bib.bib23)\]\. For atomless scores, BoN givesdN=log⁡N−\(N−1\)/Nd\_\{N\}=\\log N\-\(N\-1\)/NandRelESSN=\(2​N−1\)/N2\\mathrm\{RelESS\}\_\{N\}=\(2N\-1\)/N^\{2\}, so alreadyd2≈0\.193d\_\{2\}\\approx 0\.193\. These quantities measure pressure and support, not gain, and rules with equal KL can have different gains\. We therefore useA​d\+B​dA\\sqrt\{d\}\+Bdonly for raw\-score tilt, or for a rule with path\-specific calibration, inside a checked and reachable window\. Otherwise the practitioner lowers the pressure \(a smallerNN, mixing with the reference, or a rank\-tilt exponent\), validates the rule directly, or abstains\.

Out\-of\-distribution \(OOD\) transfer\.For cross\-slice transfer, a source priorθ¯source\\bar\{\\theta\}\_\{\\rm source\}overθ=\(A,B\)\\theta=\(A,B\)is combined with a 30% target calibration estimate asθ^shrink=λ​θ^target\+\(1−λ\)​θ¯source\\hat\{\\theta\}\_\{\\rm shrink\}=\\lambda\\hat\{\\theta\}\_\{\\rm target\}\+\(1\-\\lambda\)\\bar\{\\theta\}\_\{\\rm source\}, with a fixed target\-heavyλ=0\.75\\lambda=0\.75and calibration\-only target standardization \(Appendix[B\.12](https://arxiv.org/html/2610.41114#A2.SS12)\)\.

## 5Experiments

The experiments ask four questions in turn \(Sections[5\.1](https://arxiv.org/html/2610.41114#S5.SS1)–[5\.4](https://arxiv.org/html/2610.41114#S5.SS4)\); extended results are in Appendix[B](https://arxiv.org/html/2610.41114#A2)\.

Table 1:Prospective prediction on held\-out items \(lower is better; native units, not comparable across columns\)\. \(a\) Endpoint MAE atd=0\.10d=0\.10over valid 30/70 splits, with a 95% path\-cluster bootstrap interval for the pooledA\+BA\{\+\}BMAE; zero\-gain MAE equals the mean realized gain, and curvature sign comparessign⁡\(B^cal\)\\operatorname\{sign\}\(\\hat\{B\}\_\{\\rm cal\}\)with a coefficient fitted to the exact evaluation curve\. \(b\) Pre\-pool CurveMAE, a different metric: every method sees only the calibration items, on the common valid cells \(11 SummEval cells atd=0\.10d=0\.10are unreachable for calibration\-pool methods\)\. Paired intervals are in Appendix[B\.7](https://arxiv.org/html/2610.41114#A2.SS7)\. GPT\-4o has four paths\.### 5\.1Prospective prediction before the target pool exists

Endpoint prediction\.Atd=0\.10d=0\.10\(Table[1](https://arxiv.org/html/2610.41114#S5.T1)a\), the PACE forecast has lower endpoint MAE thanAA\-only, zero gain, andAνA\_\{\\nu\}on SummEval, eligible WMT MQM clusters, OpenMEVA\-MAGS, and the GPT\-4o judge proxy\. In the GPT\-4o setting, reweighting held\-out items toward higher judge scores yields a mean human\-score gain of0\.06800\.0680; no model is trained and no response is generated\. On SummEval and GPT\-4o, radius sweeps over0\.005≤d≤0\.200\.005\\leq d\\leq 0\.20keepA\+BA\{\+\}Bbelow the baselines, and calibration\-size curves largely stabilize by 30–50% calibration \(Appendix[B\.11](https://arxiv.org/html/2610.41114#A2.SS11)\)\.

Flexible calibration under matched information\.Table[1](https://arxiv.org/html/2610.41114#S5.T1)b compares methods that all see only the calibration split\.A\+BA\{\+\}Bhas lower CurveMAE than binned calibration in all four settings and than splines in three, and its paired intervals against splines exclude zero on SummEval and WMT\. Isotonic regression, the calibration two\-term fit, and the stored empirical curve have CurveMAE similar toA\+BA\{\+\}B, with a significant reduction only on WMT \(isotonic minusA\+BA\{\+\}B:−0\.00039\-0\.00039, 95% interval\[−0\.00070,−0\.00009\]\[\-0\.00070,\-0\.00009\]\)\. Gains from flexibility are therefore small and inconsistent: the PACE forecast is competitive rather than dominant, while remaining analytic and interpretable\.

External proxies and transfer\.With an open reward model \(Skywork\-Reward\-Llama\-3\.1\-8B\-v0\.2\[[30](https://arxiv.org/html/2610.41114#bib.bib30)\]\) or an open judge \(Qwen2\.5\-32B\-Instruct\[[38](https://arxiv.org/html/2610.41114#bib.bib38)\]\) as the SummEval proxy, addingBBreduces endpoint MAE by 27\.1% and 24\.5% relative toAA\-only, while the calibration two\-term fit is marginally lower thanA\+BA\{\+\}Bfor both \(Appendix[B\.9](https://arxiv.org/html/2610.41114#A2.SS9)\)\. For three frozen reward models as SummEval proxies, addingBBlowers the 50\-split endpoint MAE in all 12 model–dimension cells, and the calibration fit remains competitive \(Appendix[B\.10](https://arxiv.org/html/2610.41114#A2.SS10)\)\. In each split,A^​d\\hat\{A\}\\sqrt\{d\}gives the first\-order forecast andB^​d\\hat\{B\}dits curvature correction; Figure[B\.8](https://arxiv.org/html/2610.41114#A2.F8)shows the mean coherence forecasts and prediction errors over all 50 splits\. Across slices, source\-only coefficients beat zero gain on all three transfer routes, but target calibration reduces their error on every route \(e\.g\., SummEval→\\rightarrowWMT0\.0436→0\.02590\.0436\\rightarrow 0\.0259\), so source coefficients are useful priors rather than invariants \(Appendix[B\.12](https://arxiv.org/html/2610.41114#A2.SS12)\)\.

Sanity checks\.On the same sample, theory and fitted coefficients correlate at least0\.900\.90forBBand essentially one forAAacross five repeated\-rating slices \(Appendix[B\.3](https://arxiv.org/html/2610.41114#A2.SS3)\)\. SummEval human\-proxy paths have median minimum tilted variance0\.6310\.631and ESS1,3381\{,\}338ond≤0\.10d\\leq 0\.10\(Appendix[B\.15](https://arxiv.org/html/2610.41114#A2.SS15)\)\. Random and shuffled proxies driveAAand the realized gain close to zero on average, whereasAνA\_\{\\nu\}stays large \(Appendix[B\.18](https://arxiv.org/html/2610.41114#A2.SS18)\)\.

Table 2:Target\-pool\-aware CurveMAE: the proxy scores of the target pool are available, but its human labels are not\. For each calibrator, the same fittedh^\\hat\{h\}is used by projection \(the two\-term moment formula applied to\(h^​\(r\),r\)\(\\hat\{h\}\(r\),r\)on the target pool\) and by direct integration \(the exact tilt integral ofh^\\hat\{h\}\)\. The pre\-poolA\+BA\{\+\}Brow, from Table[1](https://arxiv.org/html/2610.41114#S5.T1)b, sees less information and is a reference, not a matched competitor\. Metric, support, and units as in Table[1](https://arxiv.org/html/2610.41114#S5.T1)b; paired intervals are in Appendix[B\.7](https://arxiv.org/html/2610.41114#A2.SS7)\.
### 5\.2Matched information: two\-term projection versus direct integration

When the proxy scores of the target pool are available \(Table[2](https://arxiv.org/html/2610.41114#S5.T2)\), isotonic and spline calibrators improve on the pre\-poolA\+BA\{\+\}Breference, which sees less information, except direct isotonic on GPT\-4o \(0\.008110\.00811vs\.0\.008060\.00806\); binned calibration does not\. For a fixed calibrator, projection and direct integration differ in aggregate CurveMAE by0\.000020\.00002–0\.002860\.00286across the 12 dataset–calibrator cells, and their predictions differ pointwise by0\.000410\.00041–0\.008500\.00850on average\. The choice of calibrator matters more: on SummEval, the binned projection is0\.018240\.01824above the isotonic projection\. These are descriptive differences, not uniform bounds, and an isotonich^\\hat\{h\}cannot represent a downturn\. Within the checked window, the two moment\-derived coefficients are an adequate local summary of a calibrated curve, and replacing the two\-term curve by cubic or spline fits changes CurveMAE by at most2\.1×10−52\.1\\times 10^\{\-5\}\(Appendix[B\.7](https://arxiv.org/html/2610.41114#A2.SS7)\)\.

Figure 2:What addingBBcontributes in the pre\-pool setting \(30% calibration, 50 splits per path, held\-out exact tilt gain; panel \(a\) uses the common cells of Table[1](https://arxiv.org/html/2610.41114#S5.T1)b\)\. \(a\) Relative endpoint\-MAE reduction1−MAEA\+B​\(d\)/MAEA​\-only​\(d\)1\-\\mathrm\{MAE\}\_\{A\+B\}\(d\)/\\mathrm\{MAE\}\_\{A\\text\{\-only\}\}\(d\)on identical cells, a within\-dataset ratio\. Error bars are 95% path\-cluster bootstrap intervals; GPT\-4o has 4 paths and OpenMEVA 10, and OpenMEVA’s intervals include zero ford≤0\.05d\\leq 0\.05\. \(b\) Median shareRB​\(d\)=\|B^​d\|/\(\|A^​d\|\+\|B^​d\|\)R\_\{B\}\(d\)=\|\\hat\{B\}d\|/\(\|\\hat\{A\}\\sqrt\{d\}\|\+\|\\hat\{B\}d\|\)of the predicted gain over all splits, a magnitude share rather than netBBor an error reduction\. The improvement is aggregate:A\+BA\{\+\}Bhas lower CurveMAE thanAA\-only on 55\.6–67\.0% of splits\.
### 5\.3WhatBBadds and when it is unreliable

Contribution\.In the matched pre\-pool comparison, addingB^​d\\hat\{B\}dtoA^​d\\hat\{A\}\\sqrt\{d\}reduces CurveMAE by 34\.3%, 21\.7%, 26\.4%, and 6\.4% on SummEval, GPT\-4o, WMT, and OpenMEVA \(Table[1](https://arxiv.org/html/2610.41114#S5.T1)b\)\. The OpenMEVA paired interval includes zero, so we report both predictors there\. The endpoint reduction grows withddin every setting and reaches 8–44% atd=0\.10d=0\.10\(Figure[2](https://arxiv.org/html/2610.41114#S5.F2)a\), whereas the median share ofB^​d\\hat\{B\}din the predicted gain magnitude is only 1–13% \(Figure[2](https://arxiv.org/html/2610.41114#S5.F2)b\)\. The curvature term is thus a small correction to the forecast that removes a systematic error growing with the radius\.

Reliability ofB^\\hat\{B\}\.With 30% calibration, the median coefficient\-mismatch termd​\|B^−Beval\|d\|\\hat\{B\}\-B\_\{\\rm eval\}\|atd=0\.10d=0\.10is0\.00290\.0029–0\.03170\.0317and shrinks from 5% to 30% calibration, andB^\\hat\{B\}agrees in sign with the held\-out moment coefficient on 89\.4–97\.0% of splits\. Within\-split bootstrap draws keep the sign ofB^\\hat\{B\}in 96\.7–100% of draws \(dataset medians\), yet 25–53% of split\-level intervals include zero, so sign stability does not establish nonzero curvature\. Gold quality matters: on SummEval, a second expert rating reduces the mean absolute error ofB^\\hat\{B\}from0\.06180\.0618to0\.04280\.0428, and injected rating noise removes the advantage ofA\+BA\{\+\}Bat 1\.0 rating SD for experts and 0\.5 SD for crowd workers\. Proxy\-side outliers can destabilizeB^\\hat\{B\}; after 1% positive 8\-SD proxy contamination, its correlation with the clean OpenMEVA estimate is−0\.11\-0\.11\. When the interval forBBspans zero or the scores are heavy\-tailed, report bothA\+BA\{\+\}BandAA\-only, recalibrate after robust preprocessing, or narrow the window \(Appendix[B\.8](https://arxiv.org/html/2610.41114#A2.SS8)\)\.

Table 3:LoRA\-DPO bridge: mean±\\pmsample SD over three training seeds, after averaging the 4/3/3 checkpoints per seed whose mean prompt KL is at most0\.100\.10\. Gains are in silver\-target units\.*Realized*: self\-normalized DPO/reference likelihood\-ratio reweighting of fixed pools of 32 candidates per prompt, relative to uniform; it is not a fresh\-generation, human\-rated, or exact full\-policy gain\.*Idealized*: raw\-score exponential tilt of the same pool at the same prompt radius\. Predictors use common calibration\-only coefficients\. This is a restricted predictor set; all 46 comparator variants are in Appendix[B\.16](https://arxiv.org/html/2610.41114#A2.SS16)\. Lower MAE is better\.
### 5\.4Discrete selection and actual optimization

Best\-of\-NN\.AtN=2N=2, the median induced radius \(Eq\. \([12](https://arxiv.org/html/2610.41114#S4.E12)\)\) is already outside the checked window in every setting \(0\.1660\.166–0\.1930\.193\)\. Across 191 paths and eleven values ofNNfrom 2 to 64, only 4 of 2,101 BoN points lie insided≤0\.10d\\leq 0\.10, all atN=2N=2on SummEval expert paths where 85–89% of items share the top proxy score, which caps the reachable KL\. There,A\+BA\{\+\}Boverestimates the BoN gain by0\.0200\.020on average versus0\.0880\.088forAA\-only \(descriptive,n=4n=4\)\. Prompt\-conditional BoN atN=2N=2leaves about 88% of prompts outside the window\. For smooth rules that sweep the window continuously, mixtures with BoN and rank tilts,A\+BA\{\+\}Bstays within0\.0010\.001of a calibration fit in every setting and beatsAA\-only on three of four datasets but not on OpenMEVA \(Appendix[B\.15](https://arxiv.org/html/2610.41114#A2.SS15)\)\. Because these rules do not follow raw scores, these same\-KL comparisons are descriptive scope checks\.

LoRA\-DPO bridge\.We train Qwen2\.5\-3B\-Instruct with LoRA\-DPO\[[20](https://arxiv.org/html/2610.41114#bib.bib20),[39](https://arxiv.org/html/2610.41114#bib.bib39)\]on 968 preference pairs built from UltraFeedback prompts\[[9](https://arxiv.org/html/2610.41114#bib.bib9)\], whose chosen and rejected responses are the highest\- and lowest\-scoring of eight candidates under Skywork\-Reward\-Llama\-3\.1\-8B\-v0\.2, the optimized proxyrr\. A separate frozen reward model, OpenAssistant DeBERTa\-v3\-large\[[35](https://arxiv.org/html/2610.41114#bib.bib35),[24](https://arxiv.org/html/2610.41114#bib.bib24),[19](https://arxiv.org/html/2610.41114#bib.bib19)\], supplies a silver targetJJthat is never used in training\. Coefficients are calibrated on 32 reference\-policy candidates for each of 99 usable prompts and averaged\. On 200 disjoint evaluation prompts, the policy reweights 32 fixed candidates by the normalized DPO/reference likelihood ratiowxw\_\{x\}, with radiusdx=KL\(wx∥Unif32\)d\_\{x\}=\\operatorname\{KL\}\(w\_\{x\}\\\|\\mathrm\{Unif\}\_\{32\}\)\. The realized gain in Table[3](https://arxiv.org/html/2610.41114#S5.T3)is positive but about half of the idealized tilt gain at the same radii, so the optimizer realizes only part of the local frontier\. Against realized gain,A\+BA\{\+\}Bhas lower MAE thanAA\-only in every seed and at all ten checkpoints\. Against the idealized frontier,AA\-only is closer: both forecasts underpredict it, and the negativeBBmovesA\+BA\{\+\}Btoward the optimizer’s shortfall, so part of its realized\-gain advantage is error cancellation\. Fourteen of 46 comparator variants, including flexible calibrators, have lower realized MAE \(Appendix[B\.16](https://arxiv.org/html/2610.41114#A2.SS16)\)\. This is a finite\-candidate, model\-target bridge with 8\.7% of prompt radii above 0\.10, not a human\-evaluated RLHF validation\.

## 6Discussion and limitations

Interpretation\.PACE turns a small repeated\-rating calibration matrix into a local forecast of the human\-gain curve induced by gently optimizing a fixed proxy:AAis first\-order alignment andBBthe leading curvature on the positive tilt branch, and reliability determines neither\. When target\-pool proxy scores and a flexible calibrator are available, the same two\-term summary can be read off the calibrated curve; when they are not, the PACE coefficients remain a competitive pre\-pool forecast\.

Practical use and choice of window\.Before stronger proxy pressure is applied, a practitioner \(i\) fixes the proxy, human target, reference pool, planned rule, and minimum useful gain; \(ii\) estimatesA^,B^\\hat\{A\},\\hat\{B\}and their joint bootstrap uncertainty on repeated\-rating calibration items; \(iii\) computes the induced radius, tilted variance, and RelESS of the planned rule on an unlabeled, proxy\-scored pilot; and \(iv\) chooses the largest radiusDDthat is reachable, inside the checked window, passes the support checks, and keeps a one\-sided simultaneous 95% lower gain band, including a residual allowance, above the minimum useful gain at every checkedd≤Dd\\leq D; we have not validated the coverage of such a band\. If the band crosses the threshold, the result is inconclusive; more ratings help only when sampling error, not shift, dominates\. If reachability or support fails, the pressure is reduced, the rule is validated directly, or the proxy is not used\. The window and tolerances are application\-specific;d≤0\.10d\\leq 0\.10is our checked window, not a constant\.

Cost and recalibration\.Withncaln\_\{\\rm cal\}calibration items ratedkktimes, the overhead over single labeling isncal​\(k−1\)n\_\{\\rm cal\}\(k\-1\)judgments per target dimension, paid once per calibration\. In the example, a second rating of the 300 items costs 300 extra judgments\. The ratings can be reused across proxies scored on the same items while the human target and item distribution are unchanged, and the benchmarks analyzed here already contain them\. Apart from a verified positive affine rescaling ofrr, a changed proxy, judge prompt, scoring rule, or human target requires fresh calibration\. For a shift in prompts or outputs alone, a small repeated\-rating pilot can check whether the source coefficients lead to the same decision\. The protocol is not validated without repeated ratings, and single labels or pairwise preferences are left to future work\.

Scope and reporting\.A gain forecast is interpretable only relative to the rating matrix, target definition, scalar proxy, item distribution, dimension, and checked KL window\. The theorem concerns a pooled raw\-score tilt of a fixed pool; BoN and DPO enter as bridges, and for interactive RLHF or other rules outside the checked setting, re\-estimating the coefficients alone is not sufficient\. The judge and reward\-model results concern fixed configurations, not general evaluators\. Reproducibility differs by experiment; the GPT\-4o API calls and the DPO LoRA weights are not released \(Appendix[B\.1](https://arxiv.org/html/2610.41114#A2.SS1)\)\. Appendix[B\.19](https://arxiv.org/html/2610.41114#A2.SS19)gives a reporting checklist and follow\-up checks for failures\.

## Acknowledgments and Disclosure of Funding

This research was supported by the “Advanced GPU Utilization Support Program” funded by the Government of the Republic of Korea \(Ministry of Science and ICT\) \(No\. 02\-26\-01\-0303\); by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute \(KHIDI\), funded by the Ministry of Health & Welfare, Republic of Korea \(grant number: RS\-2025\-02307233\); and by the Institute of Information & Communications Technology Planning & Evaluation \(IITP\) grant funded by the Korea government \(MSIT\) \(No\. RS\-2025\-25463302, Global Talent Recruitment for AI\-powered Drug Discovery\)\. The authors declare no competing interests\.

## References

- \[1\]Ahmadian, A\., Cremer, C\., Gallé, M\., Fadaee, M\., Kreutzer, J\., Pietquin, O\., Üstün, A\., and Hooker, S\. \(2024\)\. Back to basics: Revisiting REINFORCE\-style optimization for learning from human feedback in LLMs\. In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 12248–12267\.
- \[2\]Bai, Y\., Jones, A\., Ndousse, K\., Askell, A\., Chen, A\., DasSarma, N\., Drain, D\., Fort, S\., et al\. \(2022\)\. Training a helpful and harmless assistant with reinforcement learning from human feedback\.*arXiv preprint[arXiv:2204\.05862](https://arxiv.org/abs/2204.05862)*\.
- \[3\]Barlow, R\. E\., Bartholomew, D\. J\., Bremner, J\. M\., and Brunk, H\. D\. \(1972\)\.*Statistical inference under order restrictions: The theory and application of isotonic regression*\. John Wiley & Sons, London and New York\.
- \[4\]Beirami, A\., Agarwal, A\., Berant, J\., D’Amour, A\., Eisenstein, J\., Nagpal, C\., and Suresh, A\. T\. \(2025\)\. Theoretical guarantees on the best\-of\-n alignment policy\. In*Proceedings of the 42nd International Conference on Machine Learning*, volume 267 of*PMLR*, pages 3580–3602\.
- \[5\]Belz, A\., Thomson, C\., Reiter, E\., and Mille, S\. \(2023\)\. Non\-repeatable experiments and non\-reproducible results: The reproducibility crisis in human evaluation in NLP\. In*Findings of the Association for Computational Linguistics: ACL 2023*, pages 3676–3687\.
- \[6\]Bradley, R\. A\. and Terry, M\. E\. \(1952\)\. Rank analysis of incomplete block designs: I\. The method of paired comparisons\.*Biometrika*, 39\(3/4\):324–345\.[doi:10\.2307/2334029](https://doi.org/10.2307/2334029)\.
- \[7\]Christiano, P\. F\., Leike, J\., Brown, T\. B\., Martic, M\., Legg, S\., and Amodei, D\. \(2017\)\. Deep reinforcement learning from human preferences\. In*Advances in Neural Information Processing Systems*, volume 30\.
- \[8\]Csiszár, I\. \(1975\)\.II\-divergence geometry of probability distributions and minimization problems\.*The Annals of Probability*, 3\(1\):146–158\.[doi:10\.1214/aop/1176996454](https://doi.org/10.1214/aop/1176996454)\.
- \[9\]Cui, G\., Yuan, L\., Ding, N\., Yao, G\., He, B\., Zhu, W\., Ni, Y\., Xie, G\., et al\. \(2024\)\. UltraFeedback: Boosting language models with scaled AI feedback\. In*Proceedings of the 41st International Conference on Machine Learning*, volume 235 of*PMLR*, pages 9722–9744\.
- \[10\]Donsker, M\. D\. and Varadhan, S\. R\. S\. \(1976\)\. Asymptotic evaluation of certain Markov process expectations for large time—III\.*Communications on Pure and Applied Mathematics*, 29\(4\):389–461\.[doi:10\.1002/cpa\.3160290405](https://doi.org/10.1002/cpa.3160290405)\.
- \[11\]Eilers, P\. H\. C\. and Marx, B\. D\. \(1996\)\. Flexible smoothing with B\-splines and penalties\.*Statistical Science*, 11\(2\):89–121\.[doi:10\.1214/ss/1038425655](https://doi.org/10.1214/ss/1038425655)\.
- \[12\]Fabbri, A\. R\., Kryściński, W\., McCann, B\., Xiong, C\., Socher, R\., and Radev, D\. \(2021\)\. SummEval: Re\-evaluating summarization evaluation\.*Transactions of the Association for Computational Linguistics*, 9:391–409\.[doi:10\.1162/tacl\_a\_00373](https://doi.org/10.1162/tacl_a_00373)\.
- \[13\]Freitag, M\., Foster, G\., Grangier, D\., Ratnakar, V\., Tan, Q\., and Macherey, W\. \(2021\)\. Experts, errors, and context: A large\-scale study of human evaluation for machine translation\.*Transactions of the Association for Computational Linguistics*, 9:1460–1474\.[doi:10\.1162/tacl\_a\_00437](https://doi.org/10.1162/tacl_a_00437)\.
- \[14\]Fu, W\., Wei, B\., Hu, J\., Cai, Z\., and Liu, J\. \(2024\)\. QGEval: Benchmarking multi\-dimensional evaluation for question generation\. In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, pages 11783–11803\.
- \[15\]Gao, L\., Schulman, J\., and Hilton, J\. \(2023\)\. Scaling laws for reward model overoptimization\. In*Proceedings of the 40th International Conference on Machine Learning*, volume 202 of*PMLR*, pages 10835–10866\.
- \[16\]Goodhart, C\. A\. E\. \(1975\)\. Problems of monetary management: The U\.K\. experience\. In*Papers in Monetary Economics*, volume I\. Reserve Bank of Australia, Sydney\. Reprinted in Goodhart, C\. A\. E\. \(1984\),*Monetary Theory and Practice: The UK Experience*, pages 91–121, Macmillan, London\.
- \[17\]Guan, J\., Zhang, Z\., Feng, Z\., Liu, Z\., Ding, W\., Mao, X\., Fan, C\., and Huang, M\. \(2021\)\. OpenMEVA: A benchmark for evaluating open\-ended story generation metrics\. In*Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\)*, pages 6394–6407\.
- \[18\]Hardy, M\. \(2025\)\. “All that glitters”: Techniques for evaluations with unreliable model and human annotations\. In*Findings of the Association for Computational Linguistics: NAACL 2025*, pages 2250–2278\.
- \[19\]He, P\., Gao, J\., and Chen, W\. \(2023\)\. DeBERTaV3: Improving DeBERTa using ELECTRA\-style pre\-training with gradient\-disentangled embedding sharing\. In*International Conference on Learning Representations*\.
- \[20\]Hu, E\. J\., Shen, Y\., Wallis, P\., Allen\-Zhu, Z\., Li, Y\., Wang, S\., Wang, L\., and Chen, W\. \(2022\)\. LoRA: Low\-rank adaptation of large language models\. In*International Conference on Learning Representations*\.
- \[21\]Huang, H\., Bu, X\., Zhou, H\., Qu, Y\., Liu, J\., Yang, M\., Xu, B\., and Zhao, T\. \(2025\)\. An empirical study of LLM\-as\-a\-judge for LLM evaluation: Fine\-tuned judge model is not a general substitute for GPT\-4\. In*Findings of the Association for Computational Linguistics: ACL 2025*, pages 5880–5895\.
- \[22\]Jinnai, Y\., Morimura, T\., Ariu, K\., and Abe, K\. \(2025\)\. Regularized best\-of\-N sampling with minimum Bayes risk objective for language model alignment\. In*Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\)*, pages 9321–9347\.
- \[23\]Kong, A\. \(1992\)\. A note on importance sampling using standardized weights\. Technical Report 348, Department of Statistics, University of Chicago\.
- \[24\]Köpf, A\., Kilcher, Y\., von Rütte, D\., Anagnostidis, S\., Tam, Z\.\-R\., Stevens, K\., Barhoum, A\., Nguyen, D\., et al\. \(2023\)\. OpenAssistant Conversations – Democratizing large language model alignment\. In*Advances in Neural Information Processing Systems \(Datasets and Benchmarks Track\)*, volume 36, pages 47669–47681\.
- \[25\]Korbak, T\., Perez, E\., and Buckley, C\. L\. \(2022\)\. RL with KL penalties is better viewed as Bayesian inference\. In*Findings of the Association for Computational Linguistics: EMNLP 2022*, pages 1083–1091\.
- \[26\]Lambert, N\. and Calandra, R\. \(2023\)\. The alignment ceiling: Objective mismatch in reinforcement learning from human feedback\.*arXiv preprint[arXiv:2311\.00168](https://arxiv.org/abs/2311.00168)*\.
- \[27\]Lambert, N\., Pyatkin, V\., Morrison, J\., Miranda, L\. J\., Lin, B\. Y\., Chandu, K\., Dziri, N\., Kumar, S\., et al\. \(2025\)\. RewardBench: Evaluating reward models for language modeling\. In*Findings of the Association for Computational Linguistics: NAACL 2025*, pages 1755–1797\.
- \[28\]Lin, C\.\-Y\. \(2004\)\. ROUGE: A package for automatic evaluation of summaries\. In*Text Summarization Branches Out*, pages 74–81\. Association for Computational Linguistics\.[https://aclanthology\.org/W04\-1013/](https://aclanthology.org/W04-1013/)\.
- \[29\]Liu, Y\., Iter, D\., Xu, Y\., Wang, S\., Xu, R\., and Zhu, C\. \(2023\)\. G\-Eval: NLG evaluation using GPT\-4 with better human alignment\. In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 2511–2522\.
- \[30\]Liu, C\. Y\., Zeng, L\., Liu, J\., Yan, R\., He, J\., Wang, C\., Yan, S\., Liu, Y\., and Zhou, Y\. \(2024\)\. Skywork\-Reward: Bag of tricks for reward modeling in LLMs\.*arXiv preprint[arXiv:2410\.18451](https://arxiv.org/abs/2410.18451)*\.
- \[31\]Lommel, A\., Uszkoreit, H\., and Burchardt, A\. \(2014\)\. Multidimensional quality metrics \(MQM\): A framework for declaring and describing translation quality metrics\.*Tradumàtica: Tecnologies de la Traducció*, 12:455–463\.[doi:10\.5565/rev/tradumatica\.77](https://doi.org/10.5565/rev/tradumatica.77)\.
- \[32\]Manheim, D\. and Garrabrant, S\. \(2018\)\. Categorizing variants of Goodhart’s law\.*arXiv preprint[arXiv:1803\.04585](https://arxiv.org/abs/1803.04585)*\.
- \[33\]McGraw, K\. O\. and Wong, S\. P\. \(1996\)\. Forming inferences about some intraclass correlation coefficients\.*Psychological Methods*, 1\(1\):30–46\.[doi:10\.1037/1082\-989X\.1\.1\.30](https://doi.org/10.1037/1082-989X.1.1.30)\.
- \[34\]OpenAI \(2024\)\. GPT\-4o system card\. System card dated August 8, 2024\.[https://cdn\.openai\.com/gpt\-4o\-system\-card\.pdf](https://cdn.openai.com/gpt-4o-system-card.pdf)\.
- \[35\]OpenAssistant \(2023\)\. Reward\-model\-deberta\-v3\-large\-v2\. Model card and checkpoint, revisionc355404e\.[https://huggingface\.co/OpenAssistant/reward\-model\-deberta\-v3\-large\-v2/tree/c355404efa9ad2ad069f3a197cae0523c14244fc](https://huggingface.co/OpenAssistant/reward-model-deberta-v3-large-v2/tree/c355404efa9ad2ad069f3a197cae0523c14244fc)\.
- \[36\]Ouyang, L\., Wu, J\., Jiang, X\., Almeida, D\., Wainwright, C\. L\., Mishkin, P\., Zhang, C\., Agarwal, S\., et al\. \(2022\)\. Training language models to follow instructions with human feedback\. In*Advances in Neural Information Processing Systems*, volume 35, pages 27730–27744\.
- \[37\]Pan, A\., Bhatia, K\., and Steinhardt, J\. \(2022\)\. The effects of reward misspecification: Mapping and mitigating misaligned models\. In*International Conference on Learning Representations*\.
- \[38\]Qwen Team: Yang, A\., Yang, B\., Zhang, B\., Hui, B\., Zheng, B\., Yu, B\., Li, C\., Liu, D\., et al\. \(2024\)\. Qwen2\.5 technical report\.*arXiv preprint[arXiv:2412\.15115](https://arxiv.org/abs/2412.15115)*\.
- \[39\]Rafailov, R\., Sharma, A\., Mitchell, E\., Ermon, S\., Manning, C\. D\., and Finn, C\. \(2023\)\. Direct preference optimization: Your language model is secretly a reward model\. In*Advances in Neural Information Processing Systems*, volume 36, pages 53728–53741\.
- \[40\]Rafailov, R\., Chittepu, Y\., Park, R\., Sikchi, H\., Hejna, J\., Knox, W\. B\., Finn, C\., and Niekum, S\. \(2024\)\. Scaling laws for reward model overoptimization in direct alignment algorithms\. In*Advances in Neural Information Processing Systems*, volume 37, pages 126207–126242\.
- \[41\]Shrout, P\. E\. and Fleiss, J\. L\. \(1979\)\. Intraclass correlations: Uses in assessing rater reliability\.*Psychological Bulletin*, 86\(2\):420–428\.[doi:10\.1037/0033\-2909\.86\.2\.420](https://doi.org/10.1037/0033-2909.86.2.420)\.
- \[42\]Skalse, J\., Howe, N\. H\. R\., Krasheninnikov, D\., and Krueger, D\. \(2022\)\. Defining and characterizing reward hacking\. In*Advances in Neural Information Processing Systems*, volume 35, pages 9460–9471\.
- \[43\]Stiennon, N\., Ouyang, L\., Wu, J\., Ziegler, D\. M\., Lowe, R\., Voss, C\., Radford, A\., Amodei, D\., and Christiano, P\. F\. \(2020\)\. Learning to summarize from human feedback\. In*Advances in Neural Information Processing Systems*, volume 33, pages 3008–3021\.
- \[44\]Wang, H\., Xiong, W\., Xie, T\., Zhao, H\., and Zhang, T\. \(2024a\)\. Interpretable preferences via multi\-objective reward modeling and mixture\-of\-experts\. In*Findings of the Association for Computational Linguistics: EMNLP 2024*, pages 10582–10592\.
- \[45\]Wang, P\., Li, L\., Chen, L\., Cai, Z\., Zhu, D\., Lin, B\., Cao, Y\., Kong, L\., et al\. \(2024b\)\. Large language models are not fair evaluators\. In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 9440–9450\.
- \[46\]Yamauchi, Y\., Yano, T\., and Oyamada, M\. \(2026\)\. An empirical study of LLM\-as\-a\-judge: How design choices impact evaluation reliability\. In*Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics \(GEM\)*, pages 167–176\.
- \[47\]Zheng, L\., Chiang, W\.\-L\., Sheng, Y\., Zhuang, S\., Wu, Z\., Zhuang, Y\., Lin, Z\., Li, Z\., et al\. \(2023\)\. Judging LLM\-as\-a\-judge with MT\-Bench and Chatbot Arena\. In*Advances in Neural Information Processing Systems \(Datasets and Benchmarks Track\)*, volume 36, pages 46595–46623\.
- \[48\]Ziegler, D\. M\., Stiennon, N\., Wu, J\., Brown, T\. B\., Radford, A\., Amodei, D\., Christiano, P\., and Irving, G\. \(2019\)\. Fine\-tuning language models from human preferences\.*arXiv preprint[arXiv:1909\.08593](https://arxiv.org/abs/1909.08593)*\.

## Appendix AProofs and technical extensions

Throughout the appendix,μ=πref\\mu=\\pi\_\{\\mathrm\{ref\}\}is a probability measure and

d​πtd​μ​\(x\)=exp⁡\{t​r​\(x\)−Λ⁡\(t\)\},Λ⁡\(t\)=log⁡𝔼μ​et​r​\(x\)\.\\frac\{d\\pi\_\{t\}\}\{d\\mu\}\(x\)=\\exp\\\{tr\(x\)\-\\Lambda\(t\)\\\},\\qquad\\Lambda\(t\)=\\log\\mathbb\{E\}\_\{\\mu\}e^\{tr\(x\)\}\.WriteQ​\(t\)=𝔼πt​\[J\]Q\(t\)=\\mathbb\{E\}\_\{\\pi\_\{t\}\}\[J\],R​\(t\)=𝔼πt​\[r\]R\(t\)=\\mathbb\{E\}\_\{\\pi\_\{t\}\}\[r\], andd\(t\)=KL\(πt∥μ\)d\(t\)=\\operatorname\{KL\}\(\\pi\_\{t\}\\\|\\mu\)\. The finite\-item case used in the body of the paper is the special caseμ=Unif⁡\(𝒟\)\\mu=\\mathrm\{Unif\}\(\\mathcal\{D\}\)\. All results below specialize directly by replacingμ\\mu\-integrals with empirical sums\.

### A\.1KL\-regularized frontier proof

We prove Proposition[1](https://arxiv.org/html/2610.41114#Thmproposition1)in the more general appendix notation, replacing the finite\-item reference distributionπref\\pi\_\{\\mathrm\{ref\}\}by a probability reference measureμ\\mu\.

###### Proof of Proposition[1](https://arxiv.org/html/2610.41114#Thmproposition1)\.

Assume𝔼μ​\[\(1\+\|r\|\)​et​r\]<∞\\mathbb\{E\}\_\{\\mu\}\[\(1\+\|r\|\)e^\{tr\}\]<\\inftyand letZt=𝔼μ​et​rZ\_\{t\}=\\mathbb\{E\}\_\{\\mu\}e^\{tr\}\. Thend​πt/d​μ=et​r/Ztd\\pi\_\{t\}/d\\mu=e^\{tr\}/Z\_\{t\}has finite𝔼πt​\|r\|\\mathbb\{E\}\_\{\\pi\_\{t\}\}\|r\|and KL\. For anyπ≪μ\\pi\\ll\\muwith𝔼π​\|r\|<∞\\mathbb\{E\}\_\{\\pi\}\|r\|<\\inftyand finite KL,

KL\(π∥πt\)=𝔼πlogd​πd​μ−t𝔼πr\+logZt=KL\(π∥μ\)−t𝔼πr\+logZt\.\\operatorname\{KL\}\(\\pi\\\|\\pi\_\{t\}\)=\\mathbb\{E\}\_\{\\pi\}\\log\\frac\{d\\pi\}\{d\\mu\}\-t\\mathbb\{E\}\_\{\\pi\}r\+\\log Z\_\{t\}=\\operatorname\{KL\}\(\\pi\\\|\\mu\)\-t\\mathbb\{E\}\_\{\\pi\}r\+\\log Z\_\{t\}\.Rearranging gives

t𝔼πr−KL\(π∥μ\)=logZt−KL\(π∥πt\)≤logZt,t\\mathbb\{E\}\_\{\\pi\}r\-\\operatorname\{KL\}\(\\pi\\\|\\mu\)=\\log Z\_\{t\}\-\\operatorname\{KL\}\(\\pi\\\|\\pi\_\{t\}\)\\leq\\log Z\_\{t\},with equality iffKL\(π∥πt\)=0\\operatorname\{KL\}\(\\pi\\\|\\pi\_\{t\}\)=0, i\.e\.,π=πt\\pi=\\pi\_\{t\}almost surely\. In particular, the unconstrained optimizer is strictly unique for finitett\.

For the constrained\-frontier statement, restrict to radii in the image of an interval on whichrris non\-constant,Varπt⁡\(r\)\>0\\operatorname\{Var\}\_\{\\pi\_\{t\}\}\(r\)\>0, andd⁡\(t\)d\(t\)is strictly increasing\. The constrained primalsupπ𝔼π​\[r\]\\sup\_\{\\pi\}\\mathbb\{E\}\_\{\\pi\}\[r\]subject toKL\(π∥μ\)≤D\\operatorname\{KL\}\(\\pi\\\|\\mu\)\\leq Dhas Lagrangian𝔼π\[r\]−λ\(KL\(π∥μ\)−D\)\\mathbb\{E\}\_\{\\pi\}\[r\]\-\\lambda\(\\operatorname\{KL\}\(\\pi\\\|\\mu\)\-D\)withλ≥0\\lambda\\geq 0, and the scalarized objective in Eq\. \([3](https://arxiv.org/html/2610.41114#S3.E3)\) corresponds to the equivalent rescaling𝔼π\[r\]−t−1KL\(π∥μ\)\\mathbb\{E\}\_\{\\pi\}\[r\]\-t^\{\-1\}\\operatorname\{KL\}\(\\pi\\\|\\mu\), so the multiplier on the KL constraint isλ=1/t\\lambda=1/t\. By the standardII\-projection / Donsker–Varadhan variational argument\[[8](https://arxiv.org/html/2610.41114#bib.bib8),[10](https://arxiv.org/html/2610.41114#bib.bib10)\], the constraint is active on this regular interval, and the optimizer agrees withπt\\pi\_\{t\}\. The same uniqueness conclusion as above holds for finitett\. Specializingμ=πref\\mu=\\pi\_\{\\mathrm\{ref\}\}recovers the main\-text statement\. Degeneracy arises only at terminal radii that put all mass on the maximum\-score support, which are outside the local regime used in this paper\. ∎

### A\.2Proof of the local coefficient law

###### Theorem 2\(Local coefficient law, detailed\)\.

Assumeσr2=Varμ⁡\(r\)\>0\\sigma\_\{r\}^\{2\}=\\operatorname\{Var\}\_\{\\mu\}\(r\)\>0and eitherJ,rJ,rare bounded or the joint cumulant generating function of\(J,r\)\(J,r\)isC4C^\{4\}in a neighborhood of the origin\. Letκr=κ3​\(r,r,r\)\\kappa\_\{r\}=\\kappa\_\{3\}\(r,r,r\)andκJ​r​r=κ3​\(J,r,r\)\\kappa\_\{Jrr\}=\\kappa\_\{3\}\(J,r,r\)\. Then along the positive tilt branch,

Q⁡\(d\)=Q⁡\(0\)\+2​Covμ⁡\(J,r\)σr​d\+\(κJ​r​rσr2−2​Covμ⁡\(J,r\)​κr3​σr4\)​d\+O⁡\(d3/2\)\.Q\(d\)=Q\(0\)\+\\sqrt\{2\}\\frac\{\\operatorname\{Cov\}\_\{\\mu\}\(J,r\)\}\{\\sigma\_\{r\}\}\\sqrt\{d\}\+\\left\(\\frac\{\\kappa\_\{Jrr\}\}\{\\sigma\_\{r\}^\{2\}\}\-\\frac\{2\\operatorname\{Cov\}\_\{\\mu\}\(J,r\)\\kappa\_\{r\}\}\{3\\sigma\_\{r\}^\{4\}\}\\right\)d\+O\(d^\{3/2\}\)\.If proxy and gold share a scale andr−𝔼​r=\(J−𝔼​J\)\+εr\-\\mathbb\{E\}r=\(J\-\\mathbb\{E\}J\)\+\\varepsilon, thenCovμ⁡\(J,r\)=σJ2\+Covμ⁡\(J,ε\)\\operatorname\{Cov\}\_\{\\mu\}\(J,r\)=\\sigma\_\{J\}^\{2\}\+\\operatorname\{Cov\}\_\{\\mu\}\(J,\\varepsilon\), recovering the proxy\-error form used for interpretation\.

###### Proof\.

The proof has four steps: \(i\) expand the bivariate cumulant generating function to extractQ⁡\(t\)Q\(t\), \(ii\) expandd⁡\(t\)d\(t\)as a power series intt, \(iii\) invertd⁡\(t\)d\(t\)along the positive branch to obtaint⁡\(d\)t\(d\), and \(iv\) compose to obtainQ⁡\(d\)=Q⁡\(t⁡\(d\)\)Q\(d\)=Q\(t\(d\)\)and read off the coefficients\.

*Step 1 \(bivariate cumulant expansion ofQQ\)\.*LetG⁡\(s,t\)=log⁡𝔼μ​exp​\{s​J\+t​r\}G\(s,t\)=\\log\\mathbb\{E\}\_\{\\mu\}\\exp\\\{sJ\+tr\\\}\. The boundedness orC4C^\{4\}assumption ensuresGGisC4C^\{4\}in a neighborhood of\(0,0\)\(0,0\), so its mixed partials∂sa∂tbG⁡\(0,0\)\\partial\_\{s\}^\{a\}\\partial\_\{t\}^\{b\}G\(0,0\)fora\+b≤3a\+b\\leq 3exist and equal the joint cumulantsκa\+b​\(J⊗a,r⊗b\)\\kappa\_\{a\+b\}\(J^\{\\otimes a\},r^\{\\otimes b\}\)\. Differentiating inssats=0s=0gives the moment generating identity∂sG⁡\(0,t\)=𝔼πt​\[J\]=Q⁡\(t\)\\partial\_\{s\}G\(0,t\)=\\mathbb\{E\}\_\{\\pi\_\{t\}\}\[J\]=Q\(t\)\. Hence

Q⁡\(t\)\\displaystyle Q\(t\)=∂sG⁡\(0,t\)=∂sG⁡\(0,0\)\+t​∂t∂sG⁡\(0,0\)\+t22​∂t2∂sG⁡\(0,0\)\+RQ​\(t\)\\displaystyle=\\partial\_\{s\}G\(0,t\)=\\partial\_\{s\}G\(0,0\)\+t\\,\\partial\_\{t\}\\partial\_\{s\}G\(0,0\)\+\\tfrac\{t^\{2\}\}\{2\}\\,\\partial\_\{t\}^\{2\}\\partial\_\{s\}G\(0,0\)\+R\_\{Q\}\(t\)=𝔼μ​\[J\]\+t​Covμ⁡\(J,r\)\+t22​κ3​\(J,r,r\)\+RQ​\(t\),\\displaystyle=\\mathbb\{E\}\_\{\\mu\}\[J\]\+t\\,\\operatorname\{Cov\}\_\{\\mu\}\(J,r\)\+\\tfrac\{t^\{2\}\}\{2\}\\,\\kappa\_\{3\}\(J,r,r\)\+R\_\{Q\}\(t\),where\|RQ​\(t\)\|≤CQ​t3\|R\_\{Q\}\(t\)\|\\leq C\_\{Q\}t^\{3\}for someCQC\_\{Q\}depending only on the local fourth\-order cumulants of\(J,r\)\(J,r\)\(an explicit Taylor remainder bound under theC4C^\{4\}assumption\)\. SinceQ​\(0\)=𝔼μ​\[J\]Q\(0\)=\\mathbb\{E\}\_\{\\mu\}\[J\], this gives

Q⁡\(t\)−Q⁡\(0\)=Covμ⁡\(J,r\)​t\+12​κJ​r​r​t2\+O⁡\(t3\)\.Q\(t\)\-Q\(0\)=\\operatorname\{Cov\}\_\{\\mu\}\(J,r\)\\,t\+\\tfrac\{1\}\{2\}\\kappa\_\{Jrr\}\\,t^\{2\}\+O\(t^\{3\}\)\.
*Step 2 \(power series ford⁡\(t\)d\(t\)\)\.*Apply the same Taylor expansion toΛ⁡\(t\)=G⁡\(0,t\)\\Lambda\(t\)=G\(0,t\)inttalone:

Λ⁡\(t\)=𝔼μ​\[r\]​t\+σr22​t2\+κr6​t3\+RΛ​\(t\),\|RΛ​\(t\)\|≤CΛ​t4\.\\Lambda\(t\)=\\mathbb\{E\}\_\{\\mu\}\[r\]\\,t\+\\tfrac\{\\sigma\_\{r\}^\{2\}\}\{2\}t^\{2\}\+\\tfrac\{\\kappa\_\{r\}\}\{6\}t^\{3\}\+R\_\{\\Lambda\}\(t\),\\qquad\|R\_\{\\Lambda\}\(t\)\|\\leq C\_\{\\Lambda\}t^\{4\}\.Thend⁡\(t\)=t​Λ′​\(t\)−Λ⁡\(t\)d\(t\)=t\\Lambda^\{\\prime\}\(t\)\-\\Lambda\(t\)gives, after combining terms,

d⁡\(t\)=σr22​t2\+κr3​t3\+Rd​\(t\),\|Rd​\(t\)\|≤Cd​t4\.d\(t\)=\\tfrac\{\\sigma\_\{r\}^\{2\}\}\{2\}\\,t^\{2\}\+\\tfrac\{\\kappa\_\{r\}\}\{3\}\\,t^\{3\}\+R\_\{d\}\(t\),\\qquad\|R\_\{d\}\(t\)\|\\leq C\_\{d\}t^\{4\}\.Sinceσr2\>0\\sigma\_\{r\}^\{2\}\>0,d⁡\(t\)\>0d\(t\)\>0fort\>0t\>0small, andd′​\(t\)=t​Varπt⁡\(r\)\>0d^\{\\prime\}\(t\)=t\\operatorname\{Var\}\_\{\\pi\_\{t\}\}\(r\)\>0on a positive interval\. In particularddis strictly increasing there\.

*Step 3 \(inversion ofddalong the positive branch\)\.*Becaused⁡\(t\)∼\(σr2/2\)​t2d\(t\)\\sim\(\\sigma\_\{r\}^\{2\}/2\)t^\{2\}withσr\>0\\sigma\_\{r\}\>0, on the positive branchd1/2∼\(σr/2\)​td^\{1/2\}\\sim\(\\sigma\_\{r\}/\\sqrt\{2\}\)\\,t, suggesting the ansatz

t⁡\(d\)=a​d\+b​d\+Rt​\(d\),\|Rt​\(d\)\|≤Ct​d3/2,t\(d\)=a\\sqrt\{d\}\+b\\,d\+R\_\{t\}\(d\),\\qquad\|R\_\{t\}\(d\)\|\\leq C\_\{t\}d^\{3/2\},witha\>0a\>0\. Substituting into the expansion ford⁡\(t\)d\(t\)and matching coefficients ofddandd3/2d^\{3/2\}:

coefficient of​d:\\displaystyle\\text\{coefficient of \}d:\\quadσr22​a2=1⟹a=2σr\(positive root\),\\displaystyle\\tfrac\{\\sigma\_\{r\}^\{2\}\}\{2\}\\,a^\{2\}=1\\quad\\Longrightarrow\\quad a=\\frac\{\\sqrt\{2\}\}\{\\sigma\_\{r\}\}\\ \\ \\text\{\(positive root\)\},coefficient of​d3/2:\\displaystyle\\text\{coefficient of \}d^\{3/2\}:\\quadσr2​a​b\+κr3​a3=0⟹b=−κr3​a2σr2=−2​κr3​σr4\.\\displaystyle\\sigma\_\{r\}^\{2\}\\,a\\,b\+\\tfrac\{\\kappa\_\{r\}\}\{3\}a^\{3\}=0\\quad\\Longrightarrow\\quad b=\-\\frac\{\\kappa\_\{r\}\}\{3\}\\frac\{a^\{2\}\}\{\\sigma\_\{r\}^\{2\}\}=\-\\frac\{2\\kappa\_\{r\}\}\{3\\sigma\_\{r\}^\{4\}\}\.The negative roota=−2/σra=\-\\sqrt\{2\}/\\sigma\_\{r\}corresponds to the negative tilt branch \(decreasingEπ​\[r\]E\_\{\\pi\}\[r\]\), which is excluded by the positive\-branch hypothesis\. This determinesa,ba,buniquely\. Setx⁡\(t\)=t​d⁡\(t\)/t2x\(t\)=t\\sqrt\{d\(t\)/t^\{2\}\}, extendingd⁡\(t\)/t2d\(t\)/t^\{2\}continuously at zero\. Thenx⁡\(t\)=σr​t/2\+O⁡\(t2\)x\(t\)=\\sigma\_\{r\}t/\\sqrt\{2\}\+O\(t^\{2\}\)has nonzero derivative at zero\. Taylor inversion inx=dx=\\sqrt\{d\}yieldsRt​\(d\)=O⁡\(x3\)=O⁡\(d3/2\)R\_\{t\}\(d\)=O\(x^\{3\}\)=O\(d^\{3/2\}\); one does not apply the implicit\-function theorem directly tod−d⁡\(t\)d\-d\(t\)at zero\.

*Step 4 \(composition\)\.*Substitutet=t⁡\(d\)t=t\(d\)into the expansion forQ⁡\(t\)−Q⁡\(0\)Q\(t\)\-Q\(0\)\. The leading two terms give

Covμ⁡\(J,r\)​\[2σr​d−2​κr3​σr4​d\]\+κJ​r​r2​\[2σr​d\]2\+O⁡\(d3/2\)\.\\operatorname\{Cov\}\_\{\\mu\}\(J,r\)\\Big\[\\tfrac\{\\sqrt\{2\}\}\{\\sigma\_\{r\}\}\\sqrt\{d\}\-\\tfrac\{2\\kappa\_\{r\}\}\{3\\sigma\_\{r\}^\{4\}\}d\\Big\]\+\\tfrac\{\\kappa\_\{Jrr\}\}\{2\}\\Big\[\\tfrac\{\\sqrt\{2\}\}\{\\sigma\_\{r\}\}\\sqrt\{d\}\\Big\]^\{2\}\+O\(d^\{3/2\}\)\.The squared term isκJ​r​r2⋅2σr2​d=κJ​r​rσr2​d\\tfrac\{\\kappa\_\{Jrr\}\}\{2\}\\cdot\\tfrac\{2\}\{\\sigma\_\{r\}^\{2\}\}\\,d=\\tfrac\{\\kappa\_\{Jrr\}\}\{\\sigma\_\{r\}^\{2\}\}d\. Cross\-terms involvingd⋅d\\sqrt\{d\}\\cdot dare alreadyO⁡\(d3/2\)O\(d^\{3/2\}\)and absorbed into the remainder\. The cubic remainder\|RQ​\(t⁡\(d\)\)\|≤CQ​t​\(d\)3=O⁡\(d3/2\)\|R\_\{Q\}\(t\(d\)\)\|\\leq C\_\{Q\}t\(d\)^\{3\}=O\(d^\{3/2\}\)sincet⁡\(d\)=O⁡\(d\)t\(d\)=O\(\\sqrt\{d\}\)\. Collecting,

Q⁡\(d\)−Q⁡\(0\)=2​Covμ​\(J,r\)σr​d\+\[κJ​r​rσr2−2​Covμ⁡\(J,r\)​κr3​σr4\]​d\+O⁡\(d3/2\),Q\(d\)\-Q\(0\)=\\frac\{\\sqrt\{2\}\\,\\operatorname\{Cov\}\_\{\\mu\}\(J,r\)\}\{\\sigma\_\{r\}\}\\sqrt\{d\}\+\\Big\[\\frac\{\\kappa\_\{Jrr\}\}\{\\sigma\_\{r\}^\{2\}\}\-\\frac\{2\\operatorname\{Cov\}\_\{\\mu\}\(J,r\)\\,\\kappa\_\{r\}\}\{3\\sigma\_\{r\}^\{4\}\}\\Big\]d\+O\(d^\{3/2\}\),which gives the statedAAandBB\. ∎

### A\.3Standardized coefficient decomposition

LetZ=\(r−𝔼​r\)/σrZ=\(r\-\\mathbb\{E\}r\)/\\sigma\_\{r\}andJ¯=J−𝔼​J\\bar\{J\}=J\-\\mathbb\{E\}J\. SinceCov⁡\(J,r\)=σr​𝔼​\[J¯​Z\]\\operatorname\{Cov\}\(J,r\)=\\sigma\_\{r\}\\mathbb\{E\}\[\\bar\{J\}Z\], the alignment coefficient satisfies

A=2​Cov⁡\(J,r\)σr=2​𝔼​\[J¯​Z\]\.A=\\sqrt\{2\}\\frac\{\\operatorname\{Cov\}\(J,r\)\}\{\\sigma\_\{r\}\}=\\sqrt\{2\}\\,\\mathbb\{E\}\[\\bar\{J\}Z\]\.Moreover,

κJ​r​r=𝔼⁡\[\(J−𝔼​J\)​\(r−𝔼​r\)2\]=σr2​𝔼​\[J¯​Z2\],κr=σr3​𝔼​\[Z3\]\.\\kappa\_\{Jrr\}=\\mathbb\{E\}\[\(J\-\\mathbb\{E\}J\)\(r\-\\mathbb\{E\}r\)^\{2\}\]=\\sigma\_\{r\}^\{2\}\\mathbb\{E\}\[\\bar\{J\}Z^\{2\}\],\\qquad\\kappa\_\{r\}=\\sigma\_\{r\}^\{3\}\\mathbb\{E\}\[Z^\{3\}\]\.Substituting these identities into Theorem[1](https://arxiv.org/html/2610.41114#Thmtheorem1)gives

B=𝔼⁡\[J¯​Z2\]−23​𝔼​\[J¯​Z\]​𝔼​\[Z3\]\.B=\\mathbb\{E\}\[\\bar\{J\}Z^\{2\}\]\-\\frac\{2\}\{3\}\\mathbb\{E\}\[\\bar\{J\}Z\]\\mathbb\{E\}\[Z^\{3\}\]\.ThusAAis first\-order standardized proxy–gold alignment andBBdecomposes into a two\-sided proxy\-tail human\-quality moment and a proxy\-skewness correction\.

### A\.4Skewness correction as a change of coordinate

The skewness term inBBarises because the tilt path is indexed by the KL radius\. Lets=t​σrs=t\\sigma\_\{r\}be the standardized tilt coordinate, withZZandJ¯\\bar\{J\}as in Appendix[A\.3](https://arxiv.org/html/2610.41114#A1.SS3)and all expectations underμ\\mu\. Steps 1 and 2 of Appendix[A\.2](https://arxiv.org/html/2610.41114#A1.SS2), rewritten inss, give

Q⁡\(s\)−Q⁡\(0\)=𝔼⁡\[J¯​Z\]​s\+12​𝔼​\[J¯​Z2\]​s2\+O⁡\(s3\),d⁡\(s\)=12​s2\+13​𝔼​\[Z3\]​s3\+O⁡\(s4\)\.Q\(s\)\-Q\(0\)=\\mathbb\{E\}\[\\bar\{J\}Z\]\\,s\+\\tfrac\{1\}\{2\}\\mathbb\{E\}\[\\bar\{J\}Z^\{2\}\]\\,s^\{2\}\+O\(s^\{3\}\),\\qquad d\(s\)=\\tfrac\{1\}\{2\}s^\{2\}\+\\tfrac\{1\}\{3\}\\mathbb\{E\}\[Z^\{3\}\]\\,s^\{3\}\+O\(s^\{4\}\)\.Inverting the second expansion on the positive branch givess⁡\(d\)=2​d−23​𝔼​\[Z3\]​d\+O⁡\(d3/2\)s\(d\)=\\sqrt\{2d\}\-\\tfrac\{2\}\{3\}\\mathbb\{E\}\[Z^\{3\}\]\\,d\+O\(d^\{3/2\}\), and substituting into the first givesA=2​𝔼​\[J¯​Z\]A=\\sqrt\{2\}\\,\\mathbb\{E\}\[\\bar\{J\}Z\]andB=𝔼⁡\[J¯​Z2\]−23​𝔼​\[J¯​Z\]​𝔼​\[Z3\]B=\\mathbb\{E\}\[\\bar\{J\}Z^\{2\}\]\-\\tfrac\{2\}\{3\}\\mathbb\{E\}\[\\bar\{J\}Z\]\\mathbb\{E\}\[Z^\{3\}\]\. The first term ofBBis the second\-order quality response in the tilt coordinate\. It equalsCov⁡\(J,Z2\)\\operatorname\{Cov\}\(J,Z^\{2\}\), a two\-sided moment to which large\-\|Z\|\|Z\|items on both sides contribute, so it is not an upper\-tail\-only quantity\. The second term converts fromsstodd\. With positive alignment, right skew makes a given positivessconsume more KL, so a fixedddpermits a smallerssand the correction lowersBBrelative to its first term; left skew gives a positive correction\. The correction vanishes when either alignment or skewness is zero\.

Table[A\.1](https://arxiv.org/html/2610.41114#A1.T1)and Figure[A\.1](https://arxiv.org/html/2610.41114#A1.F1)measure when the correction matters\. The reported share,\|corr\|/\(\|𝔼⁡\[J¯​Z2\]\|\+\|corr\|\)\|\{\\rm corr\}\|/\(\|\\mathbb\{E\}\[\\bar\{J\}Z^\{2\}\]\|\+\|\{\\rm corr\}\|\)withcorr=−23​𝔼​\[J¯​Z\]​𝔼​\[Z3\]\{\\rm corr\}=\-\\tfrac\{2\}\{3\}\\mathbb\{E\}\[\\bar\{J\}Z\]\\mathbb\{E\}\[Z^\{3\}\], divides by the sum of the absolute components rather than by\|B\|\|B\|, which avoids shares above 100% when the components cancel\. In a synthetic family with a standardized skew\-normal proxy and a fixed conditional mean, the share is 0\.25% in the symmetric case and rises to about 29% for strong right skew and 52% for strong left skew\. For the full SummEval coherence pool, it is 26% with ROUGE\-1\[[28](https://arxiv.org/html/2610.41114#bib.bib28)\]and 34% with GPT\-4o as the proxy\. Changing the skew\-normal shape also changes higher moments, so the synthetic family is an intuition audit, not an isolated causal effect of skewness, and two real proxies do not establish the range of skewness found in practice\.

Table A\.1:Size of the skewness correction inBB\. Synthetic rows:ZZis a standardized skew\-normal variable with shapeaa\(20,000 items each\), with conditional mean𝔼⁡\[J∣Z\]=1\.3​Z\+0\.6​\(Z2−1\)\\mathbb\{E\}\[J\\mid Z\]=1\.3Z\+0\.6\(Z^\{2\}\-1\)and proxyr=5\+2​Zr=5\+2Z\. SummEval rows: the full pool of 1,600 coherence items with ROUGE\-1 or GPT\-4o as the proxy \(descriptive full\-pool coefficients, not calibration predictors\)\. Share is\|corr\|/\(\|𝔼⁡\[J¯​Z2\]\|\+\|corr\|\)\|\{\\rm corr\}\|/\(\|\\mathbb\{E\}\[\\bar\{J\}Z^\{2\}\]\|\+\|\{\\rm corr\}\|\), not a fraction ofBB\.Figure A\.1:When the skewness correction matters\. \(a\) Correction share as a function of proxy skewness𝔼⁡\[Z3\]\\mathbb\{E\}\[Z^\{3\}\]for the synthetic skew\-normal family \(line\) and the two SummEval coherence proxies \(points\)\. \(b\) The two components ofBBand their sum for the synthetic family: with positive alignment, right skew contributes a negative correction and left skew a positive one\. Other moments also vary across this family, so totalBBneed not decrease with skewness\.
### A\.5Finite\-range natural coordinate and global slope

###### Proposition 2\(Quadratic comparability\)\.

If0<v−≤Varπt⁡\(r\)≤v\+<∞0<v\_\{\-\}\\leq\\operatorname\{Var\}\_\{\\pi\_\{t\}\}\(r\)\\leq v\_\{\+\}<\\inftyfor allt∈\[0,T\]t\\in\[0,T\], then

v−2​t2≤d⁡\(t\)≤v\+2​t2\.\\frac\{v\_\{\-\}\}\{2\}t^\{2\}\\leq d\(t\)\\leq\\frac\{v\_\{\+\}\}\{2\}t^\{2\}\.Consequently,2/v\+​d≤t⁡\(d\)≤2/v−​d\\sqrt\{2/v\_\{\+\}\}\\sqrt\{d\}\\leq t\(d\)\\leq\\sqrt\{2/v\_\{\-\}\}\\sqrt\{d\}for attainabled∈\[0,d⁡\(T\)\]d\\in\[0,d\(T\)\]\.

###### Proof\.

Sinced⁡\(t\)=t​Λ′​\(t\)−Λ⁡\(t\)d\(t\)=t\\Lambda^\{\\prime\}\(t\)\-\\Lambda\(t\)andΛ′′​\(t\)=Varπt⁡\(r\)\\Lambda^\{\\prime\\prime\}\(t\)=\\operatorname\{Var\}\_\{\\pi\_\{t\}\}\(r\), we haved′​\(t\)=t​Varπt⁡\(r\)d^\{\\prime\}\(t\)=t\\operatorname\{Var\}\_\{\\pi\_\{t\}\}\(r\)andd⁡\(0\)=0d\(0\)=0\. Integration from00tottproves the bounds\. ∎

###### Proposition 3\(Exact global derivative identity\)\.

Assume𝔼μ​\[\(1\+\|J\|\)​\(1\+r2\)​et​r\]<∞\\mathbb\{E\}\_\{\\mu\}\[\(1\+\|J\|\)\(1\+r^\{2\}\)e^\{tr\}\]<\\inftyon\(0,tmax\)\(0,t\_\{\\max\}\)\. ThenΛ⁡\(t\)<∞\\Lambda\(t\)<\\inftyand

Q′​\(t\)=Covπt⁡\(J,r\),R′​\(t\)=Varπt⁡\(r\),d′​\(t\)=t​Varπt⁡\(r\)\.Q^\{\\prime\}\(t\)=\\operatorname\{Cov\}\_\{\\pi\_\{t\}\}\(J,r\),\\quad R^\{\\prime\}\(t\)=\\operatorname\{Var\}\_\{\\pi\_\{t\}\}\(r\),\\quad d^\{\\prime\}\(t\)=t\\operatorname\{Var\}\_\{\\pi\_\{t\}\}\(r\)\.Ifh⁡\(u\)=𝔼μ​\[J∣r=u\]h\(u\)=\\mathbb\{E\}\_\{\\mu\}\[J\\mid r=u\], then𝔼πt​\[J∣r\]=h​\(r\)\\mathbb\{E\}\_\{\\pi\_\{t\}\}\[J\\mid r\]=h\(r\)and, fort\>0t\>0,

d​Qd​d​\(d​\(t\)\)=Covπt⁡\(h⁡\(r\),r\)t​Varπt​\(r\)\\frac\{dQ\}\{dd\}\(d\(t\)\)=\\frac\{\\operatorname\{Cov\}\_\{\\pi\_\{t\}\}\(h\(r\),r\)\}\{t\\operatorname\{Var\}\_\{\\pi\_\{t\}\}\(r\)\}whenever the denominator is nonzero\.

###### Proof\.

The integrability assumption majorizes\|J​et​r\|\|Je^\{tr\}\|,\|r​et​r\|\|re^\{tr\}\|,\|J​r​et​r\|\|Jre^\{tr\}\|, andr2​et​rr^\{2\}e^\{tr\}uniformly on a neighborhood oftt, justifying differentiation under the integral\. For integrableff,𝔼πt​f=𝔼μ​\[f​et​r\]/𝔼μ​\[et​r\]\\mathbb\{E\}\_\{\\pi\_\{t\}\}f=\\mathbb\{E\}\_\{\\mu\}\[fe^\{tr\}\]/\\mathbb\{E\}\_\{\\mu\}\[e^\{tr\}\], sod​𝔼πt​f/d​t=Covπt⁡\(f,r\)d\\mathbb\{E\}\_\{\\pi\_\{t\}\}f/dt=\\operatorname\{Cov\}\_\{\\pi\_\{t\}\}\(f,r\)\. Takingf=Jf=Jandf=rf=rgives the first identities\. Sinced⁡\(t\)=t​Λ′​\(t\)−Λ⁡\(t\)d\(t\)=t\\Lambda^\{\\prime\}\(t\)\-\\Lambda\(t\), differentiating givesd′​\(t\)=t​Λ′′​\(t\)=t​Varπt⁡\(r\)d^\{\\prime\}\(t\)=t\\Lambda^\{\\prime\\prime\}\(t\)=t\\operatorname\{Var\}\_\{\\pi\_\{t\}\}\(r\)\. The density ratioet​r/Zte^\{tr\}/Z\_\{t\}is a function ofrralone, so tilting reweightsrr\-fibers without changing the conditional law ofJJgivenrr\. ∎

###### Theorem 3\(Finite\-range residual bound\)\.

FixD\>0D\>0, and letTDT\_\{D\}satisfyd⁡\(TD\)=Dd\(T\_\{D\}\)=D\. Suppose the tilt family is well\-defined on\[0,TD\]\[0,T\_\{D\}\], proxy variance is bounded above and below there, and fourth cumulants\|κ4πt​\(J,r,r,r\)\|\|\\kappa\_\{4\}^\{\\pi\_\{t\}\}\(J,r,r,r\)\|and\|κ4πt​\(r,r,r,r\)\|\|\\kappa\_\{4\}^\{\\pi\_\{t\}\}\(r,r,r,r\)\|are uniformly bounded\. LetA,BA,Bbe the coefficients in Theorem[1](https://arxiv.org/html/2610.41114#Thmtheorem1)\. Then there existsCD<∞C\_\{D\}<\\inftysuch that

\|Q⁡\(d\)−Q⁡\(0\)−A​d−B​d\|≤CD​d3/2∀d∈\[0,D\]\.\|Q\(d\)\-Q\(0\)\-A\\sqrt\{d\}\-Bd\|\\leq C\_\{D\}d^\{3/2\}\\qquad\\forall d\\in\[0,D\]\.

###### Proof\.

The result extends the local expansion to a finite checked range by combining the local residual with continuity away from zero\. We spell out the argument because it clarifies why the constant is range\-dependent\.

*Step 1 \(local residual near zero\)\.*LetK=supt∈\[0,TD\]max⁡\{\|κ4πt​\(J,r,r,r\)\|,\|κ4πt​\(r,r,r,r\)\|\}<∞K=\\sup\_\{t\\in\[0,T\_\{D\}\]\}\\max\\\{\|\\kappa\_\{4\}^\{\\pi\_\{t\}\}\(J,r,r,r\)\|,\|\\kappa\_\{4\}^\{\\pi\_\{t\}\}\(r,r,r,r\)\|\\\}<\\infty\. Taylor’s theorem with remainder applies to the marginal expansions ofQ⁡\(t\)Q\(t\)andΛ⁡\(t\)\\Lambda\(t\)on a small interval\[0,t0\]⊂\[0,TD\]\[0,t\_\{0\}\]\\subset\[0,T\_\{D\}\]\. The third and fourth derivatives are the corresponding tilted cumulants, bounded byKKon that interval\. Hence

Q⁡\(t\)−\[Q⁡\(0\)\+Covμ⁡\(J,r\)​t\+12​κJ​r​r​t2\]=O⁡\(t3\),Q\(t\)\-\\Big\[Q\(0\)\+\\operatorname\{Cov\}\_\{\\mu\}\(J,r\)\\,t\+\\tfrac\{1\}\{2\}\\kappa\_\{Jrr\}\\,t^\{2\}\\Big\]=O\(t^\{3\}\),Λ⁡\(t\)−\[𝔼μ​\[r\]​t\+σr22​t2\+κr6​t3\]=O⁡\(t4\)\.\\Lambda\(t\)\-\\Big\[\\mathbb\{E\}\_\{\\mu\}\[r\]\\,t\+\\tfrac\{\\sigma\_\{r\}^\{2\}\}\{2\}t^\{2\}\+\\tfrac\{\\kappa\_\{r\}\}\{6\}t^\{3\}\\Big\]=O\(t^\{4\}\)\.Usingd⁡\(t\)=t​Λ′​\(t\)−Λ⁡\(t\)d\(t\)=t\\Lambda^\{\\prime\}\(t\)\-\\Lambda\(t\)gives

d⁡\(t\)=σr22​t2\+κr3​t3\+O⁡\(t4\)\.d\(t\)=\\tfrac\{\\sigma\_\{r\}^\{2\}\}\{2\}t^\{2\}\+\\tfrac\{\\kappa\_\{r\}\}\{3\}t^\{3\}\+O\(t^\{4\}\)\.Sinceσr\>0\\sigma\_\{r\}\>0, the positive branch is locally invertible and

t⁡\(d\)=2σr​d−2​κr3​σr4​d\+O⁡\(d3/2\)\.t\(d\)=\\frac\{\\sqrt\{2\}\}\{\\sigma\_\{r\}\}\\sqrt\{d\}\-\\frac\{2\\kappa\_\{r\}\}\{3\\sigma\_\{r\}^\{4\}\}d\+O\(d^\{3/2\}\)\.Composing the expansion forQ⁡\(t\)Q\(t\)with this inverse gives, on0≤d≤d0:=d⁡\(t0\)0\\leq d\\leq d\_\{0\}:=d\(t\_\{0\}\),

\|Q⁡\(d\)−Q⁡\(0\)−A​d−B​d\|≤C0​d3/2\|Q\(d\)\-Q\(0\)\-A\\sqrt\{d\}\-Bd\|\\leq C\_\{0\}d^\{3/2\}for some finiteC0C\_\{0\}\.

*Step 2 \(compact tail of the checked range\)\.*Ifd0≥Dd\_\{0\}\\geq D, Step 1 proves the claim\. Otherwise, proxy variance is bounded below on\[0,TD\]\[0,T\_\{D\}\], sod⁡\(t\)d\(t\)is strictly increasing and the inverset⁡\(d\)t\(d\)is continuous on\[d0,D\]\[d\_\{0\},D\]\. Therefore

S⁡\(d\)=Q⁡\(t⁡\(d\)\)−Q⁡\(0\)−A​d−B​dS\(d\)=Q\(t\(d\)\)\-Q\(0\)\-A\\sqrt\{d\}\-Bdis continuous on the compact interval\[d0,D\]\[d\_\{0\},D\]\. Sinced3/2d^\{3/2\}is bounded away from zero there,

C1=supd∈\[d0,D\]\|S⁡\(d\)\|d3/2<∞\.C\_\{1\}=\\sup\_\{d\\in\[d\_\{0\},D\]\}\\frac\{\|S\(d\)\|\}\{d^\{3/2\}\}<\\infty\.TakingCD=max⁡\{C0,C1\}C\_\{D\}=\\max\\\{C\_\{0\},C\_\{1\}\\\}yields

\|Q⁡\(d\)−Q⁡\(0\)−A​d−B​d\|≤CD​d3/2∀d∈\[0,D\]\.\|Q\(d\)\-Q\(0\)\-A\\sqrt\{d\}\-Bd\|\\leq C\_\{D\}d^\{3/2\}\\qquad\\forall d\\in\[0,D\]\.The constantCDC\_\{D\}depends on the data distribution and on the checked range\. The theorem therefore does not assert that the two\-coefficient approximation is numerically tight for a chosen deployment radius\. Appendix[B\.15](https://arxiv.org/html/2610.41114#A2.SS15)reports finite\-range diagnostics that measure this tightness empirically\. ∎

### A\.6Calibration prediction error and finite\-sample concentration

The main text states the prediction\-error decomposition as Lemma[1](https://arxiv.org/html/2610.41114#Thmlemma1)\. The proof is immediate but useful for interpreting failures\. If

Δeval​\(d\)=Aeval​d\+Beval​d\+Reval​\(d\),\|Reval​\(d\)\|≤Ceval​d3/2,\\Delta\_\{\\rm eval\}\(d\)=A\_\{\\rm eval\}\\sqrt\{d\}\+B\_\{\\rm eval\}d\+R\_\{\\rm eval\}\(d\),\\qquad\|R\_\{\\rm eval\}\(d\)\|\\leq C\_\{\\rm eval\}d^\{3/2\},andΔ^​\(d\)=A^​d\+B^​d\\widehat\{\\Delta\}\(d\)=\\hat\{A\}\\sqrt\{d\}\+\\hat\{B\}d, then

Δ^​\(d\)−Δeval​\(d\)=\(A^−Aeval\)​d\+\(B^−Beval\)​d−Reval​\(d\)\.\\widehat\{\\Delta\}\(d\)\-\\Delta\_\{\\rm eval\}\(d\)=\(\\hat\{A\}\-A\_\{\\rm eval\}\)\\sqrt\{d\}\+\(\\hat\{B\}\-B\_\{\\rm eval\}\)d\-R\_\{\\rm eval\}\(d\)\.Taking absolute values gives Eq\. \([11](https://arxiv.org/html/2610.41114#S4.E11)\)\. The additional split into estimation error and calibration–evaluation shift is the triangle inequality, e\.g\.

\|A^−Aeval\|≤\|A^−Acal\|\+\|Acal−Aeval\|\.\|\\hat\{A\}\-A\_\{\\rm eval\}\|\\leq\|\\hat\{A\}\-A\_\{\\rm cal\}\|\+\|A\_\{\\rm cal\}\-A\_\{\\rm eval\}\|\.The sign\-margin statement follows because if the error bound is smaller than\|Δeval​\(d0\)\|\|\\Delta\_\{\\rm eval\}\(d\_\{0\}\)\|, the prediction cannot cross zero\.

#### Plug\-in coefficient estimators\.

For a calibration sample\(Ji,ri\)i=1n\(J\_\{i\},r\_\{i\}\)\_\{i=1\}^\{n\}, define empirical momentsJ¯=n−1​∑iJi\\bar\{J\}=n^\{\-1\}\\sum\_\{i\}J\_\{i\},r¯=n−1​∑iri\\bar\{r\}=n^\{\-1\}\\sum\_\{i\}r\_\{i\},σ^r2=n−1​∑i\(ri−r¯\)2\\hat\{\\sigma\}\_\{r\}^\{2\}=n^\{\-1\}\\sum\_\{i\}\(r\_\{i\}\-\\bar\{r\}\)^\{2\}, andZ^i=\(ri−r¯\)/σ^r\\hat\{Z\}\_\{i\}=\(r\_\{i\}\-\\bar\{r\}\)/\\hat\{\\sigma\}\_\{r\},J¯^i=Ji−J¯\\hat\{\\bar\{J\}\}\_\{i\}=J\_\{i\}\-\\bar\{J\}\. The plug\-in estimators corresponding to Eq\. \([6](https://arxiv.org/html/2610.41114#S3.E6)\) are

A^=2​1n​∑iJ¯^i​Z^i,B^=1n​∑iJ¯^i​Z^i2−23​\(1n​∑iJ¯^i​Z^i\)​\(1n​∑iZ^i3\)\.\\hat\{A\}=\\sqrt\{2\}\\,\\frac\{1\}\{n\}\\sum\_\{i\}\\hat\{\\bar\{J\}\}\_\{i\}\\hat\{Z\}\_\{i\},\\qquad\\hat\{B\}=\\frac\{1\}\{n\}\\sum\_\{i\}\\hat\{\\bar\{J\}\}\_\{i\}\\hat\{Z\}\_\{i\}^\{2\}\-\\frac\{2\}\{3\}\\\!\\left\(\\frac\{1\}\{n\}\\sum\_\{i\}\\hat\{\\bar\{J\}\}\_\{i\}\\hat\{Z\}\_\{i\}\\right\)\\\!\\\!\\left\(\\frac\{1\}\{n\}\\sum\_\{i\}\\hat\{Z\}\_\{i\}^\{3\}\\right\)\.Equivalently,A^=2​Cov^​\(J,r\)/σ^r\\hat\{A\}=\\sqrt\{2\}\\,\\widehat\{\\operatorname\{Cov\}\}\(J,r\)/\\hat\{\\sigma\}\_\{r\}andB^=κJ​r​r^/σ^r2−23​Cov^​\(J,r\)​κr^/σ^r4\\hat\{B\}=\\widehat\{\\kappa\_\{Jrr\}\}/\\hat\{\\sigma\}\_\{r\}^\{2\}\-\\tfrac\{2\}\{3\}\\,\\widehat\{\\operatorname\{Cov\}\}\(J,r\)\\widehat\{\\kappa\_\{r\}\}/\\hat\{\\sigma\}\_\{r\}^\{4\}, where all covariances and third\-order cumulants are the samenn\-normalized empirical central moments displayed above\. The implementation uses these plug\-in empirical moments, not finite\-sample unbiasedkk\-statistics; this convention matches the finite\-item target population used for the exact tilt curves\.

###### Proposition 4\(Coefficient concentration\)\.

In an idealized i\.i\.d\. calibration model, let\(Ji,ri\)i=1n\(J\_\{i\},r\_\{i\}\)\_\{i=1\}^\{n\}be i\.i\.d\., with\|Ji\|≤MJ\|J\_\{i\}\|\\leq M\_\{J\},\|ri\|≤Mr\|r\_\{i\}\|\\leq M\_\{r\}, and population proxy standard deviationσr≥s\>0\\sigma\_\{r\}\\geq s\>0\. Use the plug\-in estimators of Eq\. \([6](https://arxiv.org/html/2610.41114#S3.E6)\) whenσ^r\>0\\hat\{\\sigma\}\_\{r\}\>0, and setA^=B^=0\\hat\{A\}=\\hat\{B\}=0on a zero\-variance sample for this probability statement\. The empirical protocol instead excludes such samples\. Then there exist constantsC1,C2\>0C\_\{1\},C\_\{2\}\>0, depending only on\(MJ,Mr,s\)\(M\_\{J\},M\_\{r\},s\), such that for sufficiently small fixeduu,

Pr⁡\(\|A^−Acal\|\>u\)≤C1​e−C2​n​u2,Pr⁡\(\|B^−Bcal\|\>u\)≤C1​e−C2​n​u2\.\\Pr\(\|\\hat\{A\}\-A\_\{\\rm cal\}\|\>u\)\\leq C\_\{1\}e^\{\-C\_\{2\}nu^\{2\}\},\\qquad\\Pr\(\|\\hat\{B\}\-B\_\{\\rm cal\}\|\>u\)\\leq C\_\{1\}e^\{\-C\_\{2\}nu^\{2\}\}\.In particular\|A^−Acal\|,\|B^−Bcal\|=Op\(n−1/2\)\|\\hat\{A\}\-A\_\{\\rm cal\}\|,\|\\hat\{B\}\-B\_\{\\rm cal\}\|=O\_\{p\}\(n^\{\-1/2\}\)\.

###### Proof\.

The coefficients in Eq\. \([6](https://arxiv.org/html/2610.41114#S3.E6)\) are smooth functions of bounded empirical moments as long as the proxy standard deviation is bounded away from zero\. Hoeffding concentration controls each required empirical moment at raten−1/2n^\{\-1/2\}, and a delta\-method/Lipschitz argument on the event\{σ^r≥s/2\}\\\{\\hat\{\\sigma\}\_\{r\}\\geq s/2\\\}gives the displayed bounds, with constants depending on\(MJ,Mr,s\)\(M\_\{J\},M\_\{r\},s\)\. The probability of the complementary event is itself exponentially small: applying Hoeffding separately ton−1​∑iri2n^\{\-1\}\\sum\_\{i\}r\_\{i\}^\{2\}andr¯\\bar\{r\}, usingσ^r2=n−1​∑iri2−r¯2\\hat\{\\sigma\}\_\{r\}^\{2\}=n^\{\-1\}\\sum\_\{i\}r\_\{i\}^\{2\}\-\\bar\{r\}^\{2\}andσr2≥s2\\sigma\_\{r\}^\{2\}\\geq s^\{2\}, givesPr⁡\(σ^r<s/2\)≤C3​e−C4​n\\Pr\(\\hat\{\\sigma\}\_\{r\}<s/2\)\\leq C\_\{3\}e^\{\-C\_\{4\}n\}for constantsC3,C4C\_\{3\},C\_\{4\}depending on\(MJ,Mr,s\)\(M\_\{J\},M\_\{r\},s\), which can be absorbed intoC1,C2C\_\{1\},C\_\{2\}to yield the unconditional bounds stated\. ∎

###### Corollary 1\(Uniform concentration of the local prediction curve\)\.

For fixedD\>0D\>0,

sup0≤d≤D\|\(A^−Acal\)​d\+\(B^−Bcal\)​d\|=Op​\(D\+Dn\)\.\\sup\_\{0\\leq d\\leq D\}\|\(\\hat\{A\}\-A\_\{\\rm cal\}\)\\sqrt\{d\}\+\(\\hat\{B\}\-B\_\{\\rm cal\}\)d\|=O\_\{p\}\\left\(\\frac\{\\sqrt\{D\}\+D\}\{\\sqrt\{n\}\}\\right\)\.

#### Boundedness and tail robustness\.

Proposition[4](https://arxiv.org/html/2610.41114#Thmproposition4)is a finite\-sample concentration statement under bounded scores and a nondegenerate proxy variance\. The local coefficient law itself allows a local cumulant\-generating\-function condition, but the plug\-in estimator can be sensitive to extreme proxy values becauseBBcontains third\-order moments\. In finite empirical rating matrices with nonzero proxy variance the coefficients are defined after the reported preprocessing, but the diagnostic should still be accompanied by tail checks when raw proxy scores are unbounded or heavy\-tailed\. Appendix[B\.11](https://arxiv.org/html/2610.41114#A2.SS11)reports a winsorization, rank\-normalization, and controlled\-outlier audit: mild winsorization and gold\-side outliers largely preserveBBcorrelation, but sign agreement can change\. Proxy outliers can perturb curvature, motivating robust preprocessing and revalidation\.

### A\.7Best\-of\-NNas rank\-score tilt

This appendix records the rank\-score tilt formulation of best\-of\-NNused as a scope check in the body\. Recent theoretical and algorithmic analyses of BoN policies provide complementary perspectives\[[4](https://arxiv.org/html/2610.41114#bib.bib4),[22](https://arxiv.org/html/2610.41114#bib.bib22)\]\.

LetF⁡\(u\)=μ⁡\(r≤u\)F\(u\)=\\mu\(r\\leq u\)\. Ifrris atomless underμ\\mu, the probability that a sample with scorer⁡\(x\)r\(x\)is selected as the maximum amongNNiid draws is proportional toF​\(r⁡\(x\)\)N−1F\(r\(x\)\)^\{N\-1\}\. Therefore

d​πNBoNd​μ​\(x\)=N​F​\(r⁡\(x\)\)N−1=exp⁡\{\(N−1\)​log⁡F⁡\(r⁡\(x\)\)\+log⁡N\}\.\\frac\{d\\pi\_\{N\}^\{\\rm BoN\}\}\{d\\mu\}\(x\)=NF\(r\(x\)\)^\{N\-1\}=\\exp\\\{\(N\-1\)\\log F\(r\(x\)\)\+\\log N\\\}\.Thus BoN is an exponential tilt on the rank scores⁡\(x\)=log⁡F⁡\(r⁡\(x\)\)s\(x\)=\\log F\(r\(x\)\), not on the raw scorer⁡\(x\)r\(x\)\.

For the atomless KL calculation, underX∼μX\\sim\\muthe random variableU=F⁡\(r⁡\(X\)\)U=F\(r\(X\)\)is uniform on\[0,1\]\[0,1\]\. Hence

KL\(πNBoN∥μ\)=𝔼πNBoNlog\{NUN−1\}=logN\+\(N−1\)𝔼U∼Beta⁡\(N,1\)logU\.\\operatorname\{KL\}\(\\pi\_\{N\}^\{\\rm BoN\}\\\|\\mu\)=\\mathbb\{E\}\_\{\\pi\_\{N\}^\{\\rm BoN\}\}\\log\\\{NU^\{N\-1\}\\\}=\\log N\+\(N\-1\)\\mathbb\{E\}\_\{U\\sim\\mathrm\{Beta\}\(N,1\)\}\\log U\.Since𝔼Beta⁡\(N,1\)logU=−1/N\\mathbb\{E\}\_\{\\mathrm\{Beta\}\(N,1\)\}\\log U=\-1/N, we get

dNBoN=log⁡N−N−1N\.d\_\{N\}^\{\\rm BoN\}=\\log N\-\\frac\{N\-1\}\{N\}\.In particulard2BoN=log⁡2−1/2≈0\.193d\_\{2\}^\{\\rm BoN\}=\\log 2\-1/2\\approx 0\.193\.

#### Tie\-corrected discrete form\.

For discrete or tied proxy scores, letF−​\(u\)=μ⁡\(r<u\)F\_\{\-\}\(u\)=\\mu\(r<u\)\. We derive the selection density under uniform tie\-breaking among maximizers\. DrawX1,…,XNX\_\{1\},\\ldots,X\_\{N\}i\.i\.d\. fromμ\\mu, and selectX1X\_\{1\}as the BoN winner whenever \(i\) no other draw exceedsr⁡\(X1\)r\(X\_\{1\}\), and \(ii\) among draws tied withr⁡\(X1\)r\(X\_\{1\}\),X1X\_\{1\}is chosen uniformly\. Conditional only onr⁡\(X1\)=ur\(X\_\{1\}\)=u, writep:=F⁡\(u\)−F−​\(u\)p:=F\(u\)\-F\_\{\-\}\(u\)for the atom mass\. The joint event that exactlyjjof theN−1N\-1competitors tie at scoreuu, the remainingN−1−jN\-1\-jare belowuu, and none exceeduuhas probability\(N−1j\)​F−​\(u\)N−1−j​pj\\binom\{N\-1\}\{j\}F\_\{\-\}\(u\)^\{N\-1\-j\}p^\{j\}\. Uniform tie\-breaking then selectsX1X\_\{1\}with probability1/\(j\+1\)1/\(j\+1\)on that joint event\. Summing these unconditional joint probabilities gives

Pr⁡\(X1​selected∣r⁡\(X1\)=u\)=∑j=0N−1\(N−1j\)​F−​\(u\)N−1−j​pj⋅1j\+1=F​\(u\)N−F−​\(u\)NN​p,\\Pr\(X\_\{1\}\\text\{ selected\}\\mid r\(X\_\{1\}\)=u\)=\\sum\_\{j=0\}^\{N\-1\}\\binom\{N\-1\}\{j\}F\_\{\-\}\(u\)^\{N\-1\-j\}p^\{j\}\\cdot\\frac\{1\}\{j\+1\}=\\frac\{F\(u\)^\{N\}\-F\_\{\-\}\(u\)^\{N\}\}\{N\\,p\},using the binomial identity∑j=0N−1\(N−1j\)​aN−1−j​bj/\(j\+1\)=\(\(a\+b\)N−aN\)/\(N​b\)\\sum\_\{j=0\}^\{N\-1\}\\binom\{N\-1\}\{j\}a^\{N\-1\-j\}b^\{j\}/\(j\+1\)=\(\(a\+b\)^\{N\}\-a^\{N\}\)/\(Nb\)witha=F−​\(u\)a=F\_\{\-\}\(u\)andb=pb=p\. Multiplying byNN\(Bayes’ rule withPr⁡\(X1​selected\)=1/N\\Pr\(X\_\{1\}\\text\{ selected\}\)=1/N\) gives the density ratiowN​\(u\)=\(F​\(u\)N−F−​\(u\)N\)/\(F⁡\(u\)−F−​\(u\)\)w\_\{N\}\(u\)=\\big\(F\(u\)^\{N\}\-F\_\{\-\}\(u\)^\{N\}\\big\)/\\big\(F\(u\)\-F\_\{\-\}\(u\)\\big\)on the atom casep\>0p\>0\. Lettingp↓0p\\downarrow 0recoverswN​\(u\)=N​F​\(u\)N−1w\_\{N\}\(u\)=NF\(u\)^\{N\-1\}\. Hence

wN​\(u\)=\{F​\(u\)N−F−​\(u\)NF​\(u\)−F−​\(u\),F⁡\(u\)\>F−​\(u\),N​F​\(u\)N−1,F​\(u\)=F−​\(u\),w\_\{N\}\(u\)=\\begin\{cases\}\\dfrac\{F\(u\)^\{N\}\-F\_\{\-\}\(u\)^\{N\}\}\{F\(u\)\-F\_\{\-\}\(u\)\},&F\(u\)\>F\_\{\-\}\(u\),\\\\\[6\.00006pt\] NF\(u\)^\{N\-1\},&F\(u\)=F\_\{\-\}\(u\),\\end\{cases\}andd​πNBoN/d​μ​\(x\)=wN​\(r⁡\(x\)\)d\\pi\_\{N\}^\{\\rm BoN\}/d\\mu\(x\)=w\_\{N\}\(r\(x\)\)\. The atomless formula is recovered whenμ\\muhas no atoms inrr\.

#### Relative ESS, softened rules, and saturation\.

For a rule\-induced distributionppon a pool with referencep0p\_\{0\},RelESS\(p,p0\)=\(∑ipi2/p0,i\)−1=1/\{1\+χ2\(p∥p0\)\}∈\(0,1\]\\mathrm\{RelESS\}\(p,p\_\{0\}\)=\(\\sum\_\{i\}p\_\{i\}^\{2\}/p\_\{0,i\}\)^\{\-1\}=1/\\\{1\+\\chi^\{2\}\(p\\\|p\_\{0\}\)\\\}\\in\(0,1\]\. For a uniform reference onmmitems it equalsESS/m\\mathrm\{ESS\}/mwithESS=1/∑ipi2\\mathrm\{ESS\}=1/\\sum\_\{i\}p\_\{i\}^\{2\}, so the effective number of candidates ism​RelESSm\\,\\mathrm\{RelESS\}\. For atomless scores,𝔼μ​\[wN2\]=∫01N2​u2​N−2​𝑑u=N2/\(2​N−1\)\\mathbb\{E\}\_\{\\mu\}\[w\_\{N\}^\{2\}\]=\\int\_\{0\}^\{1\}N^\{2\}u^\{2N\-2\}\\,du=N^\{2\}/\(2N\-1\), soRelESSN=\(2​N−1\)/N2\\mathrm\{RelESS\}\_\{N\}=\(2N\-1\)/N^\{2\}, which is0\.7500\.750,0\.4380\.438, and0\.2340\.234atN=2,4,8N=2,4,8, wheredN=0\.193d\_\{N\}=0\.193,0\.6360\.636, and1\.2041\.204\. For atomless scores,qλ∝μ​F​\(r\)λq\_\{\\lambda\}\\propto\\mu F\(r\)^\{\\lambda\}has density\(λ\+1\)​Fλ=wλ\+1\(\\lambda\+1\)F^\{\\lambda\}=w\_\{\\lambda\+1\}and equals BoN whenN=λ\+1N=\\lambda\+1is an integer\. The finite\-pool experiments instead useUi=rankavg⁡\(ri\)/mU\_\{i\}=\\operatorname\{rank\}\_\{\\rm avg\}\(r\_\{i\}\)/m, with one\-based average ranks for ties, andqλ,i∝p0,i​Uiλq\_\{\\lambda,i\}\\propto p\_\{0,i\}U\_\{i\}^\{\\lambda\}\. This empirical rank tilt need not equal finite\-pool BoN, even without ties\. The mixturepα=\(1−α\)​μ\+α​πNBoNp\_\{\\alpha\}=\(1\-\\alpha\)\\mu\+\\alpha\\pi^\{\\rm BoN\}\_\{N\}reaches arbitrarily small radii and satisfiesχ2\(pα∥μ\)=α2χ2\(πNBoN∥μ\)\\chi^\{2\}\(p\_\{\\alpha\}\\\|\\mu\)=\\alpha^\{2\}\\chi^\{2\}\(\\pi^\{\\rm BoN\}\_\{N\}\\\|\\mu\), henceRelESS\(pα\)=1/\{1\+α2χ2\(πNBoN∥μ\)\}\\mathrm\{RelESS\}\(p\_\{\\alpha\}\)=1/\\\{1\+\\alpha^\{2\}\\chi^\{2\}\(\\pi^\{\\rm BoN\}\_\{N\}\\\|\\mu\)\\\}; its KL has no closed form and is computed exactly\. Ties also cap the radius\. AsN→∞N\\to\\infty, BoN and the pooled tilt both converge to the uniform distribution on thentopn\_\{\\rm top\}items tied at the maximum score, sod≤log⁡\(m/ntop\)d\\leq\\log\(m/n\_\{\\rm top\}\), and a proxy with many tied top scores keeps BoN at small radii even for moderateNN\(Appendix[B\.15](https://arxiv.org/html/2610.41114#A2.SS15)\)\. All of these quantities are computable from unlabeled proxy scores\. Lying inside the checked window is necessary but not sufficient, because rules with equal KL can have different gains\.

#### Pooled versus prompt\-conditional optimization\.

The exponential\-tilt target in the main theorem is a pooled finite\-item tilt over the reference distribution supplied to the diagnostic\. If items are pairs\(x,y\)\(x,y\)with promptxxand candidate responseyy, the pooled tilt

πt​\(x,y\)∝πref​\(x,y\)​exp⁡\{t​r​\(x,y\)\}\\pi\_\{t\}\(x,y\)\\propto\\pi\_\{\\rm ref\}\(x,y\)\\exp\\\{tr\(x,y\)\\\}can change the prompt marginal because prompts with larger partition functions receive more mass\. Standard RLHF, reranking, and BoN deployments more often apply proxy pressure conditionally on each prompt,

πt​\(y∣x\)∝πref​\(y∣x\)​exp⁡\{t​r​\(x,y\)\},πt​\(x\)=πref​\(x\),\\pi\_\{t\}\(y\\mid x\)\\propto\\pi\_\{\\rm ref\}\(y\\mid x\)\\exp\\\{tr\(x,y\)\\\},\\qquad\\pi\_\{t\}\(x\)=\\pi\_\{\\rm ref\}\(x\),which equals the pooled tilt only when the prompt\-level normalizer is constant across prompts or when the reference distribution is already defined on a fixed prompt\-conditional slice\. The paper therefore treats the smooth pooled tilt as the theorem target and reports prompt\-conditional BoN as a scope audit rather than as a direct theorem validation; Table[B\.46](https://arxiv.org/html/2610.41114#A2.T46)and Figure[B\.13](https://arxiv.org/html/2610.41114#A2.F13)quantify this distinction\.

### A\.8Measurement aggregation toy theorem

LetYi​j=Ji\+ηi​jY\_\{ij\}=J\_\{i\}\+\\eta\_\{ij\}withηi​j\\eta\_\{ij\}i\.i\.d\. acrossjj, independent ofJiJ\_\{i\}, with𝔼​η=0\\mathbb\{E\}\\eta=0andVar⁡\(η\)=τ2\\operatorname\{Var\}\(\\eta\)=\\tau^\{2\}\. Assume finite third moments andσJ2\+τ2\>0\\sigma\_\{J\}^\{2\}\+\\tau^\{2\}\>0\. Let thekk\-rater proxy berk=J\+η¯kr\_\{k\}=J\+\\bar\{\\eta\}\_\{k\}, whereη¯k=k−1​∑j=1kηi​j\\bar\{\\eta\}\_\{k\}=k^\{\-1\}\\sum\_\{j=1\}^\{k\}\\eta\_\{ij\}\.

###### Proposition 5\(First and second moments under aggregation\)\.

For this additive\-noise model withσJ2\+τ2\>0\\sigma\_\{J\}^\{2\}\+\\tau^\{2\}\>0,

σrk2=σJ2\+τ2/k,Cov⁡\(J,rk\)=σJ2\.\\sigma\_\{r\_\{k\}\}^\{2\}=\\sigma\_\{J\}^\{2\}\+\\tau^\{2\}/k,\\qquad\\operatorname\{Cov\}\(J,r\_\{k\}\)=\\sigma\_\{J\}^\{2\}\.Therefore

Ak=2​σJ2σJ2\+τ2/kA\_\{k\}=\\sqrt\{2\}\\frac\{\\sigma\_\{J\}^\{2\}\}\{\\sqrt\{\\sigma\_\{J\}^\{2\}\+\\tau^\{2\}/k\}\}is nondecreasing inkk, strictly increasing when bothτ2\>0\\tau^\{2\}\>0andσJ2\>0\\sigma\_\{J\}^\{2\}\>0, and converges to2​σJ\\sqrt\{2\}\\,\\sigma\_\{J\}\.

###### Proposition 6\(Third cumulants under aggregation\)\.

LetκJ=κ3​\(J,J,J\)\\kappa\_\{J\}=\\kappa\_\{3\}\(J,J,J\)andκη=κ3​\(η,η,η\)\\kappa\_\{\\eta\}=\\kappa\_\{3\}\(\\eta,\\eta,\\eta\)\. Under the i\.i\.d\. rater\-noise assumption above,

κ3​\(J,rk,rk\)=κJ,κ3​\(rk,rk,rk\)=κJ\+κη/k2\.\\kappa\_\{3\}\(J,r\_\{k\},r\_\{k\}\)=\\kappa\_\{J\},\\qquad\\kappa\_\{3\}\(r\_\{k\},r\_\{k\},r\_\{k\}\)=\\kappa\_\{J\}\+\\kappa\_\{\\eta\}/k^\{2\}\.Consequently

Bk=κJσrk2−2​σJ2​\(κJ\+κη/k2\)3​σrk4,B\_\{k\}=\\frac\{\\kappa\_\{J\}\}\{\\sigma\_\{r\_\{k\}\}^\{2\}\}\-\\frac\{2\\sigma\_\{J\}^\{2\}\(\\kappa\_\{J\}\+\\kappa\_\{\\eta\}/k^\{2\}\)\}\{3\\sigma\_\{r\_\{k\}\}^\{4\}\},which need not improve monotonically withkk\.

The toy calculation matches the intervention experiments: rater aggregation reliably improves reliability andAA, but local headroom also depends on curvature\.

### A\.9ICC random\-effects clarification

We follow the standard intraclass\-correlation conventions of[Shrout and Fleiss \[41\]](https://arxiv.org/html/2610.41114#bib.bib41)and[McGraw and Wong \[33\]](https://arxiv.org/html/2610.41114#bib.bib33)\. A standard random\-effects model writes

Yi​j=μ\+Ui\+Vj\+Wi​j,Y\_\{ij\}=\\mu\+U\_\{i\}\+V\_\{j\}\+W\_\{ij\},whereUi∼\(0,σU2\)U\_\{i\}\\sim\(0,\\sigma\_\{U\}^\{2\}\)is item\-level quality,Vj∼\(0,σV2\)V\_\{j\}\\sim\(0,\\sigma\_\{V\}^\{2\}\)is rater bias, andWi​j∼\(0,σW2\)W\_\{ij\}\\sim\(0,\\sigma\_\{W\}^\{2\}\)is residual interaction/noise\. The three components are mutually independent\. Two single\-rater reliability conventions appear in the body:

ICC⁡\(A,1\)=σU2σU2\+σV2\+σW2,ICC⁡\(C,1\)=σU2σU2\+σW2,\\mathrm\{ICC\}\(A,1\)=\\frac\{\\sigma\_\{U\}^\{2\}\}\{\\sigma\_\{U\}^\{2\}\+\\sigma\_\{V\}^\{2\}\+\\sigma\_\{W\}^\{2\}\},\\qquad\\mathrm\{ICC\}\(C,1\)=\\frac\{\\sigma\_\{U\}^\{2\}\}\{\\sigma\_\{U\}^\{2\}\+\\sigma\_\{W\}^\{2\}\},that is, ICC\(A,1\) is the absolute\-agreement form \(treatingVjV\_\{j\}as variance to be explained\), and ICC\(C,1\) is the consistency form \(rater bias absorbed/conditioned out\)\. The correspondingkk\-rater versions replaceσW2\\sigma\_\{W\}^\{2\}byσW2/k\\sigma\_\{W\}^\{2\}/k\(and, for ICC\(A,1\),σV2\\sigma\_\{V\}^\{2\}byσV2/k\\sigma\_\{V\}^\{2\}/kwhen raters are crossed and averaged\)\. Thus, schematically,

ICC⁡\(C,k\)=σU2σU2\+σW2/k,ICC⁡\(A,k\)=σU2σU2\+σV2/k\+σW2/k\.\\mathrm\{ICC\}\(C,k\)=\\frac\{\\sigma\_\{U\}^\{2\}\}\{\\sigma\_\{U\}^\{2\}\+\\sigma\_\{W\}^\{2\}/k\},\\qquad\\mathrm\{ICC\}\(A,k\)=\\frac\{\\sigma\_\{U\}^\{2\}\}\{\\sigma\_\{U\}^\{2\}\+\\sigma\_\{V\}^\{2\}/k\+\\sigma\_\{W\}^\{2\}/k\}\.We useνk2≈Relk\\nu\_\{k\}^\{2\}\\approx\\mathrm\{Rel\}\_\{k\}as a generic reliability anchor\. In concrete tables,Relk\\mathrm\{Rel\}\_\{k\}is taken to be the reported ICC variant\. ICC\(A,1\) is the default absolute\-agreement reliability anchor\. ICC\(C,1\) is reported separately to diagnose rater\-bias or personalization gaps, not as the defaultAνA\_\{\\nu\}estimator\.

In the prospective direct\-proxy tables, the operational reliability\-only baseline is the plug\-in surrogate

Aν=2​σ^J​\(σ^J/σ^r\),A\_\{\\nu\}=\\sqrt\{2\}\\,\\hat\{\\sigma\}\_\{J\}\(\\hat\{\\sigma\}\_\{J\}/\\hat\{\\sigma\}\_\{r\}\),computed on the calibration split for each proxy–gold path\. This is the empirical analogue ofν2=σJ2/σr2\\nu^\{2\}=\\sigma\_\{J\}^\{2\}/\\sigma\_\{r\}^\{2\}in the simple noise\-only modelr=J\+ηr=J\+\\etawithη⟂J\\eta\\perp J, which corresponds to ICC\(C,1\) after identifyingσU2\\sigma\_\{U\}^\{2\}withσJ2\\sigma\_\{J\}^\{2\}andσW2\\sigma\_\{W\}^\{2\}withVar⁡\(η\)\\operatorname\{Var\}\(\\eta\)\. When an ICC estimate is converted to a reliability amplitude, we use the square root of its nonnegative part\.

### A\.10Qualitative coefficient readout

Table[A\.2](https://arxiv.org/html/2610.41114#A1.T2)lists qualitative interpretations of the local coefficient law in Eq\. \([4](https://arxiv.org/html/2610.41114#S3.E4)\) for representative coefficient patterns\. The readout is local\. Finite\-range deployment still requires the reachability, tilted\-variance, and ESS checks of Section[4](https://arxiv.org/html/2610.41114#S4)\.

Table A\.2:Qualitative readout of the local coefficient law\. The table is only a local interpretation of Eq\. \([4](https://arxiv.org/html/2610.41114#S3.E4)\)\. Finite\-range deployment requires the reachability, tilted\-variance, and ESS checks of Section[4](https://arxiv.org/html/2610.41114#S4)\.

## Appendix BExperimental details and additional results

This appendix gives the experiment\-level details behind the main tables and figures\. The goal is not to add new headline claims, but to make the repeated\-rating construction, admissibility filters, robustness checks, and failure analyses inspectable\.

### B\.1Reproducibility package and asset provenance

Code, prepared inputs, and derived results are released at[https://github\.com/pacifiic/local\-proxy\-diagnostics](https://github.com/pacifiic/local-proxy-diagnostics)\. The release regenerates the tables and figures of this paper from saved results; large files are stored compressed with recorded checksums, and a verification script checks the saved numerical outputs\. Reproducibility differs by experiment, and we distinguish three levels\. \(i\)*GPT\-4o judge\.*The released aggregate proxy scores reproduce all downstream analyses, but not the original API calls: an exact rerun would require raw per\-call logs with the model snapshot, decoding parameters, and invalid\-output and retry counts, which the aggregate does not contain \(Appendix[B\.14](https://arxiv.org/html/2610.41114#A2.SS14)\)\. \(ii\)*Open reward models and open judge*\(Appendices[B\.9](https://arxiv.org/html/2610.41114#A2.SS9)and[B\.10](https://arxiv.org/html/2610.41114#A2.SS10)\)\. The scored files reproduce all downstream analyses, and the model identifiers, resolved revisions, and scoring settings are recorded; we do not claim byte\-level replay of the upstream scoring runs\. \(iii\)*LoRA\-DPO bridge*\(Appendix[B\.16](https://arxiv.org/html/2610.41114#A2.SS16)\)\. The pinned candidate pools and scores, code, seed schedule, and derived summaries are released, but the LoRA weights were not retained\.

Large public source datasets are not redistributed when their terms require users to obtain them from the original providers; the release lists the download commands, pinned source revisions, and preparation steps\. All resampling uses explicit seeds\. Each experiment family records its numerical environment\. Results are reproduced in that environment; for example, the cross\-validation tie order of a few DPO comparators can change selected hyperparameters on other hardware \(Appendix[B\.16](https://arxiv.org/html/2610.41114#A2.SS16)\)\.

#### Compute\.

The experimental node had two Intel Xeon 6960P processors \(144 physical cores / 288 threads\), approximately 2\.2 TiB of system RAM, and eight NVIDIA B200 GPUs \(approximately 179 GiB of device memory each\)\. Human\-rating analyses, including flexible calibration, coefficient stability, and selection\-rule audits, are CPU\-only\. The logged endpoint, transfer, and validation reproduction took approximately 4\.25 wall\-clock hours, with another 0\.5 hours for the robustness checks of Appendix[B\.11](https://arxiv.org/html/2610.41114#A2.SS11)\. These mostly single\-process Python jobs used about 5–10 CPU\-hours in total and less than 32 GB of RAM\. Candidate generation, reward\-model and judge scoring, and LoRA\-DPO training used one GPU per worker, allowing independent jobs to run in parallel\. Each of the three reward\-model scorers in Appendix[B\.10](https://arxiv.org/html/2610.41114#A2.SS10)used one B200\.

#### GPU runtimes\.

Generating the shared 17,600 DPO candidates with Qwen2\.5\-3B\-Instruct took approximately 3\.5 minutes on one GPU \(0\.058 GPU\-hours\)\. The three\-seed DPO sweep used six separately trained step budgets per seed \(64, 90, 120, 150, 180, and 200\), totaling 1\.216 GPU\-hours of training\-loop time across 18 jobs, including those later excluded by the KL gate\. This excludes model loading and likelihood\-ratio evaluation\. The five\-pass Qwen2\.5\-32B\-Instruct judge run produced 32,000 SummEval judgments in approximately 18\.5 minutes on one GPU \(0\.309 GPU\-hours, tensor parallelism 1\)\. Generation and judge timings include engine initialization and shutdown\. These logged timings cover the stated stages, rather than the full project GPU cost\.

Table[B\.1](https://arxiv.org/html/2610.41114#A2.T1)lists the source, license or access terms where verified, and redistribution status for each external asset; when we could not verify a public license page for the source used here, the table marks the terms as not specified\.

Table B\.1:Asset provenance, license/terms, and redistribution status\. Raw external datasets are cited in the paper and obtained from their original sources unless the reproducibility package explicitly includes a derived data product\. Users should follow the source license or access terms for each external asset\.

### B\.2Datasets, repeated\-rating construction, and admissibility

The clean theorem\-validation experiments require repeated ratings on the same item\. For an itemiiand held\-out raterjj, the proxy isri=Yi​jr\_\{i\}=Y\_\{ij\}and the gold target is the leave\-one\-rater\-out meanJi=1mi−1​∑j′≠jYi​j′J\_\{i\}=\\frac\{1\}\{m\_\{i\}\-1\}\\sum\_\{j^\{\\prime\}\\neq j\}Y\_\{ij^\{\\prime\}\}\. This rule is deliberately restrictive: splits without repeated segment–system ratings are excluded because they cannot support a human proxy and a disjoint human gold signal on the same item\. Table[B\.2](https://arxiv.org/html/2610.41114#A2.T2)summarizes the analysis\-eligible slices, which retain only the systems, language pairs, story sources, raters, and dimensions that support finite\-item path construction, leave\-one\-rater\-out or disjoint human gold, and repeated calibration/evaluation resampling\. The 11 skipped SummEval attempts are finite\-sample path failures, not post\-hoc outcome filtering\.

Table B\.2:Experimental protocol and split accounting for the endpoint and transfer analyses\. Paths are proxy–gold tilt paths, and valid attempts are calibration/evaluation resamples used for prospective prediction atd=0\.10d=0\.10\. Human\-proxy rows use repeated ratings to construct both the proxy view and the human target, and the GPT\-4o row uses an external judge score as the proxy and repeated SummEval human ratings as the target\. Labels name analysis\-eligible slices or transfer routes, not entire source benchmarks\.#### Pairwise\-preference data\.

Pairwise\-preference datasets do not directly satisfy this requirement\. They provide comparisons rather than repeated scalar observations of the same item\-level quality, so using them would require an additional utility model, such as a Bradley–Terry or Thurstone construction, beforeJiJ\_\{i\}andrir\_\{i\}can be put on a scalar finite\-item tilt path\. Such utilities are useful engineering summaries, but they add structural assumptions that are outside the nonparametric repeated\-rating validation protocol\. We therefore use pairwise\-derived resources only as auxiliary geometry checks unless repeated scalar human ratings are available\.

#### SummEval\.

SummEval\[[12](https://arxiv.org/html/2610.41114#bib.bib12)\]contains 1,600 evaluated summaries, four dimensions, and two rater pools: five crowd raters and three expert raters\. Table[B\.3](https://arxiv.org/html/2610.41114#A2.T3)reports the ICC\-style reliability decomposition used as the verifiability anchor\. Reliability varies sharply by dimension and pool\. Expert reliability is much higher than crowd reliability for consistency and fluency, moderately higher for coherence, and slightly lower for relevance under ICC\(A,1\)\. This heterogeneity is why the paper treats verifiability as an observation\-process quantity rather than as a benchmark constant\.

Table B\.3:SummEval reliability and score statistics\.GpersG\_\{\\rm pers\}is the gap between consistency\-style and absolute\-agreement reliability\.Table B\.4:Expert–crowd reliability lift on SummEval\. The expert lift is dimension\-dependent rather than uniform\.![Refer to caption](https://arxiv.org/html/2610.41114v1/fig1_summeval_icc_by_dimension.png)Figure B\.1:SummEval reliability by dimension and rater pool\. The plot visualizes Table[B\.3](https://arxiv.org/html/2610.41114#A2.T3): reliability is a property of the observation protocol, not a single scalar attached to the benchmark\. Error bars, where shown, are bootstrap intervals for the ICC estimate under the split construction\.![Refer to caption](https://arxiv.org/html/2610.41114v1/fig3_expert_vs_crowd_lift.png)Figure B\.2:Expert–crowd reliability lift\. Consistency and fluency show large measurement gains from expert raters\. Relevance does not, which motivates separating measurement improvement from preference heterogeneity\. Error bars show bootstrap intervals for the expert\-minus\-crowd ICC difference\.
#### WMT MQM/pSQM admissibility\.

The raw WMT inventory\[[13](https://arxiv.org/html/2610.41114#bib.bib13),[31](https://arxiv.org/html/2610.41114#bib.bib31)\]contains many one\-rating\-per\-item splits\. These are useful for benchmark evaluation, but not for our direct human tilt construction\. Table[B\.5](https://arxiv.org/html/2610.41114#A2.T5)lists the WMT splits that pass the repeated\-rating filter and therefore enter the direct\-tilt analysis\. For MQM, 8 of 19 inspected splits provide repeated ratings and yield 61 held\-out\-rater paths\. For pSQM, both inspected 2020 language\-pair splits pass the filter and yield 12 paths\. The main prospective WMT row uses a stricter prepared MQM cluster matrix with 47 complete rater\-set clusters and 141 held\-out\-rater paths\. It should be read as eligible WMT MQM clusters, not all WMT MQM\.

#### WMT preprocessing correction\.

The prospective cluster preparation treats quotation marks literally in the tab\-delimited source files and includes document IDs in segment keys, which prevents dropped or merged records and collisions between document\-local segment IDs\. An earlier preparation lacked both safeguards and produced 45 clusters and 139 paths instead of 47 and 141; every WMT result in this paper uses the corrected input, which lowers the endpointA\+BA\{\+\}BMAE from 0\.0958 to 0\.0846 and preserves the ordering of the predictors\. The separate raw\-data direct\-validation loader already used document\-aware keys and literal\-tab parsing, so its 61 MQM and 12 pSQM paths are unaffected\. The two pipelines also use different scoring conventions: the prepared clusters use severity weights, whereas raw MQM validation additionally distinguishes error categories\. Their path counts therefore describe different analysis slices\.

Table B\.5:WMT repeated\-rating splits eligible to the direct\-tilt analysis\. Splits with one rating per segment–system item are excluded by construction\.FamilySplitPathscorr\(AA\)corr\(BB\)sign agreemedian itemsICC\(1,1\)MQMgeneralMT2022\_\_ende\_\_mqm\_\_3r70\.99990\.96277/784600\.488MQMgeneralMT2022\_\_enzh\_\_mqm\_\_raw60\.99990\.94606/6132400\.417MQMgeneralMT2023\_\_ende\_\_mqm\_\_3r100\.99950\.960610/1017550\.638MQMgeneralMT2023\_\_ende\_\_mqm\_\_sxs\_3r100\.99980\.956410/103100\.499MQMgeneralMT2023\_\_zhen\_\_mqm\_\_3r80\.99930\.90618/870560\.579MQMgeneralMT2023\_\_zhen\_\_mqm\_\_sxs\_3r80\.99980\.97698/814800\.457MQMnewstest2020\_\_ende\_\_mqm\_\_raw60\.99690\.91076/671100\.410MQMnewstest2020\_\_zhen\_\_mqm\_\_raw61\.00000\.97836/699450\.556pSQMnewstest2020\_\_ende\_\_psqm\_\_raw61\.00000\.92376/670300\.535pSQMnewstest2020\_\_zhen\_\_psqm\_\_raw61\.00000\.99976/699250\.098

#### OpenMEVA, QGEval, and auxiliary sources\.

For OpenMEVA\-MAGS ROC/WP annotations\[[17](https://arxiv.org/html/2610.41114#bib.bib17)\], we use only the manually annotated human\-rating files because the diagnostic needs repeated scalar ratings on the same story–model item\. Other OpenMEVA benchmark components are outside this analysis\. Within OpenMEVA, the prospective, OOD\-transfer, and calibration\-size analyses use the 10\-path ROC/WP slice, while the direct\-validation appendix tables report pooled, ROC, and WP scalar\-rating paths where noted\. QGEval\-human\[[14](https://arxiv.org/html/2610.41114#bib.bib14)\]is used only where repeated human dimensions permit direct coefficient validation\. RewardBench and MT\-Bench are auxiliary geometry checks: RewardBench uses cross\-model score margins, and MT\-Bench uses split\-half Bradley–Terry utilities rather than repeated scalar human ratings\.

Figure B\.3:ICC reliability profiles across the repeated\-rating analysis slices\. Panel A shows SummEval dimension\-level ICC\(A,1\), with mean ICC≈0\.22\\approx 0\.22\. Panel B shows one\-way ICC\(1,1\) for eligible raw WMT MQM splits, with mean ICC≈0\.51\\approx 0\.51; these are the direct\-validation splits, not the prepared prospective clusters\. Panel C shows OpenMEVA\-MAGS reliability, with mean ICC≈0\.75\\approx 0\.75\. The dashed line in each panel marks the slice mean\.

### B\.3Direct coefficient\-law validation and robustness

The main text reports pooled direct\-tilt validation\. This subsection expands the direct SummEval and robustness results\. Table[B\.6](https://arxiv.org/html/2610.41114#A2.T6)gives the rendered multi\-dataset validation row that supports the direct\-validation claim, including the QGEval\-human repeated scalar\-rating slice\. For OpenMEVA\-MAGS, the direct\-validation row pools the pooled, ROC, and WP scalar\-rating splits, with 5 paths from each split for 15 total paths\. In the default SummEval LOO construction, all 32 held\-out\-rater paths have negative theoretical and fittedBB, with mean localR2R^\{2\}essentially equal to one\. The strongest expert dimensions, consistency and fluency, also have the most negative curvature, so improved measurement does not automatically imply larger local headroom\.

Table B\.6:Human direct\-tilt validation of the local coefficient law\. All fits use exact exponential tilt and LOO/disjoint human gold where applicable\. The QGEval\-human row is the direct scalar\-human validation slice listed in Table[B\.2](https://arxiv.org/html/2610.41114#A2.T2)\.Figure B\.4:SummEval exact\-tilt validation example\. One human rater defines the proxy and LOO ratings defineJJ\. The checked window is shaded throughd=0\.10d=0\.10; the dash\-dot line marks the nominal fitting cutoffd=0\.20d=0\.20\. Curves stop at the computed exact path neard=0\.15d=0\.15, and the unattainable region is shaded gray\.Table B\.7:SummEval LOO direct\-tilt coefficient summary by rater pool and dimension atd≤0\.1d\\leq 0\.1\.The coefficient recovery is not a fragile consequence of choosingd≤0\.1d\\leq 0\.1\. Table[B\.8](https://arxiv.org/html/2610.41114#A2.T8)shows the exact\-tilt validation over wider fit windows on the eight SummEval pool–dimension groups\. Using reachable points below each nominal cutoff, theAAcorrelation is at least0\.99860\.9986and theBBcorrelation exceeds0\.9820\.982, with8/88/8sign agreement\. Only 2 of 32 paths can reachd=2\.0d=2\.0\. The main windowd≤0\.1d\\leq 0\.1lies below the atomlessN=2N=2BoN radius; tied finite pools can have smaller radii\.

Table B\.8:SummEval fit\-range sensitivity for direct exact\-tilt validation\.The default estimator is also robust to score standardization and to using the median of the other raters as gold\. Table[B\.9](https://arxiv.org/html/2610.41114#A2.T9)fixes the fit window atd≤0\.1d\\leq 0\.1\. Same\-mean gold is included only as a mechanical sanity check\. It is not the preferred validation because it reuses the same mean structure on both sides\. The LOO and median\-other variants give nearly identical conclusions\.

Table B\.9:Estimator sensitivity atd≤0\.1d\\leq 0\.1on SummEval\.#### Ordinal\-artifact check\.

A possible concern is that discretizing continuous scores into a small number of rating levels could itself induce negativeBB\. The ordinal simulation does not support this as the main explanation\. Across 5\-point and 9\-point synthetic scales with noise standard deviations from 0\.25 to 1\.25, the continuous baseline keepsPr⁡\(Bfit<0\)\\Pr\(B\_\{\\rm fit\}<0\)near one half, whereas the ordinal transformation raises it only to about0\.490\.49–0\.680\.68, depending on scale and noise\. Thus, discretization can shift curvature modestly, but the effect is far weaker than the near\-universal negative\-BBpattern in SummEval, QGEval\-human, and eligible WMT MQM paths\.

### B\.4Cross\-slice and feedback\-geometry extensions

The main text summarizes the repeated scalar human\-rating validation as the cleanest theorem\-checking regime\. Table[B\.10](https://arxiv.org/html/2610.41114#A2.T10)records broader extensions that are useful but should not be read as equally controlled theorem validation\. MT\-Bench\[[47](https://arxiv.org/html/2610.41114#bib.bib47)\]uses split\-half Bradley–Terry utilities from pairwise judgments\[[6](https://arxiv.org/html/2610.41114#bib.bib6)\]\. QGEval metric\-proxy paths use automatic metrics asrragainst human gold\[[14](https://arxiv.org/html/2610.41114#bib.bib14)\]\. RewardBench uses a pinned recursive continuous score\-margin snapshot fromallenai/reward\-bench\-resultsrather than human repeated ratings\[[27](https://arxiv.org/html/2610.41114#bib.bib27)\]\. For RewardBench we use pairwise\-available rows, so each model path uses the score\-margin items available for that model instead of dropping a whole subset when another model is missing a margin\. The common pattern is that coefficient recovery remains strong, but the sign regime is more heterogeneous outside the repeated scalar human setup\.

Table B\.10:Feedback\-geometry and proxy\-regime extensions\. These are breadth checks, not the main human direct\-rating validation regime\.OpenMEVA\-MAGS\[[17](https://arxiv.org/html/2610.41114#bib.bib17)\]is the clearest human\-rating heterogeneity case among the eligible story\-generation annotations\. The pooled split has 4/5 sign agreement with no negative theoreticalBB\. In the ROC split all 5 paths have non\-negative theoretical and fittedBB, while the WP split is mixed:2/52/5theoretical paths and3/53/5fitted paths have negativeBB\. This supports the paper’s central wording: the coefficient law transfers, but the local sign regime is empirical\.

Table B\.11:OpenMEVA\-MAGS pooled/ROC/WP direct\-tilt summary by split\. The table does not cover the full OpenMEVA benchmark suite\.
### B\.5ExactAAversus normalizedAνA\_\{\\nu\}

The exact local law predictsA=2​\(σJ2\+ρ\)/σrA=\\sqrt\{2\}\(\\sigma\_\{J\}^\{2\}\+\\rho\)/\\sigma\_\{r\}\. In the direct human\-proxy experiments, the simplified reliability\-only coefficientAν=2​σJ​νA\_\{\\nu\}=\\sqrt\{2\}\\sigma\_\{J\}\\nuis valid only when the correlated proxy\-error termρ\\rhois negligible and the reliability quantityν\\nucoincides with the trained\-proxy signal ratioσJ/σr\\sigma\_\{J\}/\\sigma\_\{r\}under the same measurement model\. Table[B\.12](https://arxiv.org/html/2610.41114#A2.T12)shows that this extra condition is not empirically satisfied: the median\|ρ\|/σJ2\|\\rho\|/\\sigma\_\{J\}^\{2\}is0\.2120\.212on SummEval and0\.3380\.338on eligible WMT MQM clusters\. The exact coefficient still matches the fitted coefficient essentially perfectly, whileAνA\_\{\\nu\}is noticeably rougher and induces large surrogate\-headroom error\.

For the GPT\-4o external\-proxy setting, the reportedAνA\_\{\\nu\}row is not an independently estimated proxy\-rater ICC baseline\. The GPT scoring passes are first averaged into one item\-level proxy scorerir\_\{i\}\. On each calibration split, we then compute the same variance\-ratio surrogate used in the direct\-proxy experiments,

A^ν=2​σ^J​\(σ^J/σ^r\)=2​σ^J2/σ^r,\\widehat\{A\}\_\{\\nu\}=\\sqrt\{2\}\\,\\hat\{\\sigma\}\_\{J\}\(\\hat\{\\sigma\}\_\{J\}/\\hat\{\\sigma\}\_\{r\}\)=\\sqrt\{2\}\\,\\hat\{\\sigma\}\_\{J\}^\{2\}/\\hat\{\\sigma\}\_\{r\},whereσ^J2\\hat\{\\sigma\}\_\{J\}^\{2\}andσ^r2\\hat\{\\sigma\}\_\{r\}^\{2\}are the calibration\-split item\-level variances of the human aggregate target and the aggregated GPT\-4o proxy score\. The stored number of GPT scoring passes \(njudgesn\_\{\\rm judges\}in Table[B\.41](https://arxiv.org/html/2610.41114#A2.T41)\) is used only to form the item\-level mean proxy score and does not enter theAνA\_\{\\nu\}baseline separately\. Thus, in the external\-proxy experiment,AνA\_\{\\nu\}should be interpreted as a scale/reliability\-style surrogate rather than as a measured GPT\-rater ICC\.

Table B\.12:ExactAAcoefficient versus normalizedAνA\_\{\\nu\}approximation\. The “smallρ\\rho” threshold is\|ρ\|/σJ2≤0\.1\|\\rho\|/\\sigma\_\{J\}^\{2\}\\leq 0\.1\.
### B\.6Prospective prediction by group

Table[B\.13](https://arxiv.org/html/2610.41114#A2.T13)compares the coefficient predictor with an empirical two\-parameter fit and other baselines\. The empirical fit estimatesAAandBBby least squares on the exact calibration\-split gain curve over0<d≤0\.100<d\\leq 0\.10, then predicts the evaluation gain without using evaluation labels\. It is a usable data\-driven baseline under the same calibration protocol\. The oracle instead uses evaluation moment coefficients and is only a retrospective diagnostic of finite\-range approximation error\. The moment\-based predictor has similar endpoint MAE to the empirical fit while providing closed\-form coefficient estimates with an analytic interpretation\.

Table[B\.14](https://arxiv.org/html/2610.41114#A2.T14)gives the compact multi\-slice view used in the main text\. Prospective gain and MAE use each human target’s native score units, including MQM penalties for WMT\. Absolute MAEs are therefore not directly comparable across datasets\. Within the eligible WMT slice, the coefficient predictor has lower MAE thanAA\-only, zero\-gain, and reliability\-only baselines\.

Table B\.13:Prospective prediction baselines atd=0\.10d=0\.10, including the calibration\-only empirical\-fit baseline and retrospective oracle\. Brackets are 95% path\-cluster bootstrap intervals for pooled\-split MAE \(1,000 draws, seed 0\)\. Each draw retains all valid splits of its sampled paths\. Split rows sharing paths/items make these descriptive uncertainty summaries rather than iid split\-level intervals\. For the GPT\-4o row,AνA\_\{\\nu\}denotes the calibration\-split variance\-ratio surrogate computed after averaging GPT scoring passes to one item\-level proxy score; no GPT\-rater ICC is estimated\.Table B\.14:Prospective prediction summary across the eligible analysis slices used in the main text\. WMT means complete\-rater MQM clusters\. OpenMEVA means MAGS ROC/WP human annotations\.The path/repeat/valid\-split accounting for the headline prospective rows is in Table[B\.2](https://arxiv.org/html/2610.41114#A2.T2)\. Figure[B\.5](https://arxiv.org/html/2610.41114#A2.F5)summarizes the endpoint comparison and one transfer route graphically\.

Figure B\.5:Overview of endpoint prediction and OOD transfer\. Panel A compares endpoint gain\-prediction error atd=0\.10d=0\.10for SummEval human\-rater proxy paths and the GPT\-4o judge proxy experiment\. Panel B shows the SummEval→\\rightarrowWMT OOD transfer comparison under the stricterz\_calprotocol\. Source\-prior denotes the fixed target\-heavy shrinkage estimator\. Error bars are 95% path\-cluster bootstrap intervals for pooled\-split MAE; GPT\-4o has only four dimension paths\. Lower MAE is better\.Table[B\.15](https://arxiv.org/html/2610.41114#A2.T15)localizes the prospective result usingBB\-sign agreement, not fixed\-gain sign accuracy\. The human\-proxy expert groups have perfectBB\-sign agreement, while Turker consistency and fluency are harder coefficient\-sign cases\. Their magnitude MAE remains small\. In GPT\-4o, all four dimensions retain positive mean gain atd=0\.10d=0\.10, with coherence largest at0\.08650\.0865and fluency smallest at0\.04460\.0446\.BB\-sign agreement is9090–100%100\\%by dimension, and theory MAE stays between0\.00970\.0097and0\.01260\.0126\.

Table B\.15:Prospective prediction diagnostics by group\. Resamples are valid item splits\. BothBB\-sign columns use the evaluation curve\-fit coefficient as their reference\.Table B\.16:Audit separating the two sign metrics used in the paper\.BB\-sign agreement compares the calibration moment coefficient sign with the sign ofBevalfitB\_\{\\rm eval\}^\{\\rm fit\}, fitted to the exact evaluation gain curve over0<d≤0\.100<d\\leq 0\.10\. Fixed\-gain sign accuracy compares the predicted and actual gain signs atd=0\.10d=0\.10\.Figure B\.6:Prospective prediction diagnostics across eligible human\-proxy analysis slices\. Panel A comparesA\+BA\{\+\}BMAE against the reliability\-only baseline \(AνA\_\{\\nu\}\) atd=0\.10d=0\.10on SummEval, WMT MQM clusters, and OpenMEVA\-MAGS ROC/WP; the brokenyy\-axis separates the high\-errorAνA\_\{\\nu\}/zero baselines from the lower coefficient errors\. Panel B reports agreement between the calibration moment coefficient sign and the evaluation curve\-fit coefficient sign,sign⁡\(B^cal\)=sign⁡\(Bevalfit\)\\operatorname\{sign\}\(\\hat\{B\}\_\{\\rm cal\}\)=\\operatorname\{sign\}\(B\_\{\\rm eval\}^\{\\rm fit\}\), together with mean realized gainΔ⁡\(0\.10\)\\Delta\(0\.10\)for the same human\-proxy slices\. The evaluation coefficient is fitted over0<d≤0\.100<d\\leq 0\.10and is used only for retrospective scoring\. The 50% and 90% lines are visual references, not calibrated acceptance thresholds\. GPT\-4o results are in Tables[B\.13](https://arxiv.org/html/2610.41114#A2.T13)and[B\.15](https://arxiv.org/html/2610.41114#A2.T15)\.Table B\.17:Margin\-conditioned fixed\-gain sign accuracy atd=0\.10d=0\.10\. The Condition column thresholds the actual held\-out gain margin\|Δ⁡\(0\.10\)\|\|\\Delta\(0\.10\)\|, so this is an oracle diagnostic of sign\-margin sensitivity rather than a deployable selection rule\. The human\-proxy case has positive gains throughout\. GPT\-4o proxy retains perfect sign accuracy across the realized\-margin sweep until the most aggressive filter leaves no retained cases\.
### B\.7Flexible calibration under matched information

#### Calibrators and tuning\.

All calibrators are fitted on the calibration items of each split\. The two information regimes share the split plan, seeds, and code, and differ only in the pool over which the fittedh^\\hat\{h\}is integrated\. Isotonic regression is nondecreasing and untuned\. Binned calibration uses equal\-frequency proxy bins, with candidate bin countsb∈\{3,5,8,12\}b\\in\\\{3,5,8,12\\\}restricted by3​b≤ncal3b\\leq n\_\{\\rm cal\}\(ties can leave smaller bins\), and shrinks bin means toward the calibration mean with pseudo\-countk∈\{0,5,20\}k\\in\\\{0,5,20\\\}; both are chosen by five\-fold cross\-validated squared error\. The spline uses a cubic B\-spline basis with constant extrapolation and\{4,6,8\}\\\{4,6,8\\\}knots chosen by five\-fold cross\-validation, with a ridge penalty chosen by efficient leave\-one\-out over 13 log\-spaced values in\[10−3,103\]\[10^\{\-3\},10^\{3\}\]\. A small multilayer perceptron with two hidden ReLU layers of 16 units is pre\-specified, untuned, and reported only as exploratory\. The calibration two\-term fit samples the exact calibration\-pool tilt curve at 200 equally spaced tilt values up to KLmin⁡\(0\.10,0\.95​dmax\)\\min\(0\.10,\\,0\.95\\,d\_\{\\max\}\), wheredmaxd\_\{\\max\}is the largest KL reachable by the calibration pool, and fitsa​d\+b​da\\sqrt\{d\}\+bdwithout intercept\.

#### Support\.

Each path has 50 random 30/70 splits \(seed 0\), with 480/1,120 items for SummEval and GPT\-4o, 300/700 for OpenMEVA, and a median of 267/623 for the WMT clusters\. WMT uses the corrected 141\-path input throughout\. Methods that integrate over the calibration pool are undefined when that pool cannot reach the requested radius\. This happens only for 11 SummEval cells atd=0\.10d=0\.10, all on one expert consistency path whose calibration splits reach at most KL0\.0800\.080–0\.0980\.098\. Every comparison uses the cells valid for all compared methods \(7,989 of 8,000 for SummEval\)\. With this support, the SummEval endpoint row atd=0\.10d=0\.10of Table[B\.19](https://arxiv.org/html/2610.41114#A2.T19)reproduces the endpoint MAE in Table[1](https://arxiv.org/html/2610.41114#S5.T1)a\.

#### Paired comparisons\.

Table[B\.18](https://arxiv.org/html/2610.41114#A2.T18)reports the pre\-pool differences fromA\+BA\{\+\}B\. The two\-term fit, the stored curve, and isotonic regression are significantly better thanA\+BA\{\+\}Bonly on WMT; on the four\-path GPT\-4o slice,A\+BA\{\+\}Bis lower than isotonic with an interval that barely excludes zero\. Table[B\.19](https://arxiv.org/html/2610.41114#A2.T19)gives endpoint MAE at each radius, and Table[B\.20](https://arxiv.org/html/2610.41114#A2.T20)gives the target\-pool\-aware differences, of which only projection minus direct integration is information\-matched\. Across the 12 dataset–calibrator cells, the aggregate projection–direct gap is0\.000020\.00002–0\.002860\.00286and the mean pointwise gap is0\.000410\.00041–0\.008500\.00850\. These are different quantities, and neither is a uniform bound from the theorem\.

#### Functional form\.

In a separate run on all cells of each slice, replacing the fitted two\-term calibration curve by cubic, six\-knot spline, or residual\-spline curves fitted to the same calibration data changes CurveMAE by at most2\.1×10−52\.1\\times 10^\{\-5\}on every dataset\. This supports the empirical adequacy of the two\-term form on this grid, not its theoretical dominance\.

Table B\.18:Pre\-pool CurveMAE differences fromA\+BA\{\+\}B\(method minusA\+BA\{\+\}B; positive values mean thatA\+BA\{\+\}Bhas lower error\), on the common valid cells of Table[1](https://arxiv.org/html/2610.41114#S5.T1)b\. Intervals are 95% percentile path\-clustered bootstrap intervals \(2,000 draws, paths resampled, seed 0\), not adjusted for multiplicity\.†Recomputed with the same protocol because the saved interval used all 8,000 SummEval cells\. GPT\-4o has 4 paths, so its intervals are descriptive\. The MLP is exploratory\.Table B\.19:Pre\-pool endpoint MAE at each radius, pooled over the common valid cells at that radius \(SummEval uses 1,589 of 1,600 cells atd=0\.10d=0\.10for every method\)\. This is endpoint MAE, not CurveMAE; itsd=0\.10d=0\.10column is comparable to Table[1](https://arxiv.org/html/2610.41114#S5.T1)a\. Native units; lower is better\. The MLP is exploratory\.Table B\.20:Target\-pool\-aware paired CurveMAE differences \(first\-named minus second\-named; negative values favor the first\)\. Only projection minus direct integration compares the sameh^\\hat\{h\}with the same information; comparisons with the pre\-poolA\+BA\{\+\}Banchor mix information regimes\. The mean pointwise gap is the path\-mean of\|projection−direct\|\|\\text\{projection\}\-\\text\{direct\}\|predictions, a disagreement rather than an error\. Intervals as in Table[B\.18](https://arxiv.org/html/2610.41114#A2.T18);†computed with the same protocol from row\-level predictions\.

### B\.8Contribution and reliability ofBB

#### Contribution\.

Tables[B\.21](https://arxiv.org/html/2610.41114#A2.T21)and[B\.22](https://arxiv.org/html/2610.41114#A2.T22)give the numbers behind Figure[2](https://arxiv.org/html/2610.41114#S5.F2)\. Both use the pre\-pool protocol with 30% calibration\. Error reductions use the common valid cells of Table[1](https://arxiv.org/html/2610.41114#S5.T1)b; magnitude shares use all path\-splits \(1,600 for SummEval, versus 1,589 common cells atd=0\.10d=0\.10\)\. The endpoint reduction grows with the radius in every dataset; OpenMEVA’s interval includes zero ford≤0\.05d\\leq 0\.05, and its CurveMAE interval includes zero\. The shareRB​\(d\)=\|B^​d\|/\(\|A^​d\|\+\|B^​d\|\)R\_\{B\}\(d\)=\|\\hat\{B\}d\|/\(\|\\hat\{A\}\\sqrt\{d\}\|\+\|\\hat\{B\}d\|\)is a magnitude share of the predicted gain\. It grows liked\\sqrt\{d\}, because\|B^​d\|/\|A^​d\|=\(\|B^\|/\|A^\|\)​d\|\\hat\{B\}d\|/\|\\hat\{A\}\\sqrt\{d\}\|=\(\|\\hat\{B\}\|/\|\\hat\{A\}\|\)\\sqrt\{d\}, and it says nothing about accuracy\. The aggregate improvement coexists with split\-level heterogeneity:A\+BA\{\+\}Bhas lower CurveMAE thanAA\-only on 55\.6–67\.0% of path\-split units\.

#### Coefficient error and sign stability\.

Table[B\.23](https://arxiv.org/html/2610.41114#A2.T23)a decomposes the error atd=0\.10d=0\.10with the terms of Lemma[1](https://arxiv.org/html/2610.41114#Thmlemma1)\. HereAevalA\_\{\\rm eval\}andBevalB\_\{\\rm eval\}are moment coefficients computed on the held\-out split and used only retrospectively, sod​\|A^−Aeval\|\\sqrt\{d\}\|\\hat\{A\}\-A\_\{\\rm eval\}\|andd​\|B^−Beval\|d\|\\hat\{B\}\-B\_\{\\rm eval\}\|combine calibration estimation error with calibration–evaluation shift, and the remainder is the finite\-range error of the held\-out curve’s own two\-term approximation\. The sign agreement in this panel is*moment\-based*: it comparesB^cal\\hat\{B\}\_\{\\rm cal\}withBevalB\_\{\\rm eval\}and gives 89\.4–97\.0% at 30% calibration\. The 90\.7–97\.5% in Table[1](https://arxiv.org/html/2610.41114#S5.T1)a instead comparesB^cal\\hat\{B\}\_\{\\rm cal\}with a coefficient fitted to the exact held\-out curve\. Table[B\.23](https://arxiv.org/html/2610.41114#A2.T23)b uses 300 cluster\-bootstrap draws within each calibration split\. The sign ofB^\\hat\{B\}is kept in a dataset median of 96\.7–100% of draws, yet 25\.0–52\.6% of split\-level 95% intervals include zero, so sign stability does not establish nonzero population curvature\. With an operational threshold of0\.1​σ^J0\.1\\,\\hat\{\\sigma\}\_\{J\}on the lower bootstrap percentile of the predicted gain,AA\-only andA\+BA\{\+\}Bmake different proceed decisions on 25\.5% of GPT\-4o splits and on at most 2\.5% of splits elsewhere\.

#### Rater count and rating noise\.

Table[B\.24](https://arxiv.org/html/2610.41114#A2.T24)and Figure[B\.7](https://arxiv.org/html/2610.41114#A2.F7)vary only the calibration gold on SummEval: the proxy is a fixed held\-out human rater, and evaluation gold is always the clean mean of all non\-proxy raters\. One to two expert gold raters reduce the mean absoluteBBerror from0\.06180\.0618to0\.04280\.0428and CurveMAE from0\.02690\.0269to0\.01440\.0144; one to four crowd raters reduce them from0\.06590\.0659to0\.05900\.0590and from0\.01220\.0122to0\.01040\.0104\. Under injected Gaussian or variance\-normalized Student\-t3t\_\{3\}noise, scaled by the group rating SD and clipped to the observed range, the sign ofB^\\hat\{B\}agrees with the clean estimate in at least 99\.3% \(experts\) and 93\.0% \(crowd\) of splits\. Its predictive usefulness is less stable: the advantage ofA\+BA\{\+\}BoverAA\-only disappears at 1\.0 rating SD for experts and 0\.5 SD for crowd workers\. These CurveMAE values come from their own split streams and are not comparable with Table[1](https://arxiv.org/html/2610.41114#S5.T1)b\.

#### Heavy tails\.

Table[B\.36](https://arxiv.org/html/2610.41114#A2.T36)in Appendix[B\.11](https://arxiv.org/html/2610.41114#A2.SS11)audits preprocessing and contamination with full\-pool coefficients\. Injecting 1% positive 8\-SD outliers into the gold scores leavesBBcorrelated 0\.955–0\.997 with the clean estimates \(sign agreement 93\.0–98\.6%\), but contaminating the proxy destabilizes OpenMEVA most \(correlation−0\.107\-0\.107, sign agreement 20\.0%\)\. This bounded stress test does not establish robustness to unbounded heavy\-tailed scores outside the regularity assumptions of Theorem[1](https://arxiv.org/html/2610.41114#Thmtheorem1)\. When tails are heavy, we recommend inspecting them, recalibrating after robust preprocessing, narrowing the window, or abstaining\.

Table B\.21:AA\-only versusA\+BA\{\+\}Bin the pre\-pool protocol \(common valid cells\)\. \(a\) Endpoint MAE at each radius, with the relative reduction1−MAEA\+B/MAEA​\-only1\-\\mathrm\{MAE\}\_\{A\+B\}/\\mathrm\{MAE\}\_\{A\\text\{\-only\}\}and its 95% path\-cluster bootstrap interval \(2,000 draws, seed 0\)\. \(b\) CurveMAE with the paired differenceA\+BA\{\+\}BminusAA\-only \(negative favorsA\+BA\{\+\}B\) and the share of path\-split units won byA\+BA\{\+\}B; paths/cells are 32/7,989 \(SummEval\), 4/1,000 \(GPT\-4o\), 141/35,250 \(WMT\), and 10/2,500 \(OpenMEVA\)\. MAEs are in native units and are not comparable across datasets; GPT\-4o intervals rest on four paths\.\(a\) Endpoint MAE by radiusDatasetddCellsAA\-onlyA\+BA\{\+\}BA\+B−AA\{\+\}B\-A\-onlyRed\. \(%\)95% CI \(%\)SummEval0\.0101,6000\.00630\.0057−\-0\.00069\.2\[5\.9, 13\.3\]0\.0251,6000\.01080\.0088−\-0\.002018\.6\[12\.0, 24\.9\]0\.0501,6000\.01760\.0123−\-0\.005330\.2\[19\.8, 38\.2\]0\.0751,6000\.02430\.0150−\-0\.009338\.1\[25\.9, 46\.7\]0\.1001,5890\.03100\.0174−\-0\.013644\.0\[30\.8, 52\.8\]GPT\-4o0\.0102000\.00420\.0039−\-0\.00037\.2\[4\.1, 10\.7\]0\.0252000\.00690\.0060−\-0\.000912\.5\[6\.8, 18\.5\]0\.0502000\.01040\.0084−\-0\.002019\.0\[10\.3, 27\.4\]0\.0752000\.01350\.0102−\-0\.003324\.4\[13\.1, 34\.5\]0\.1002000\.01650\.0118−\-0\.004728\.7\[14\.9, 40\.0\]WMT0\.0107,0500\.03230\.0297−\-0\.00257\.9\[6\.9, 8\.9\]0\.0257,0500\.05320\.0454−\-0\.007814\.7\[13\.0, 16\.5\]0\.0507,0500\.08080\.0620−\-0\.018823\.3\[20\.7, 25\.9\]0\.0757,0500\.10580\.0743−\-0\.031529\.8\[26\.7, 32\.9\]0\.1007,0500\.12980\.0846−\-0\.045234\.8\[31\.5, 38\.2\]OpenMEVA0\.0105000\.00460\.0045−\-0\.00011\.9\[−\-1\.1, 4\.6\]0\.0255000\.00750\.0072−\-0\.00033\.7\[−\-0\.7, 7\.7\]0\.0505000\.01090\.0103−\-0\.00065\.9\[−\-0\.2, 11\.3\]0\.0755000\.01370\.0127−\-0\.00107\.4\[0\.1, 13\.8\]0\.1005000\.01620\.0148−\-0\.00138\.3\[0\.1, 15\.5\]\(b\) CurveMAE overd∈\{0\.01,0\.025,0\.05,0\.075,0\.10\}d\\in\\\{0\.01,0\.025,0\.05,0\.075,0\.10\\\}CurveMAEA\+B−AA\{\+\}B\-A\-onlyUnitsDatasetAA\-onlyA\+BA\{\+\}BDiff\.95% CIRed\. \(%\)won \(%\)SummEval0\.018030\.01185−\-0\.00618\[−\-0\.00962,−\-0\.00324\]34\.367\.0GPT\-4o0\.010300\.00806−\-0\.00224\[−\-0\.00364,−\-0\.00100\]21\.765\.5WMT0\.080390\.05921−\-0\.02118\[−\-0\.02451,−\-0\.01826\]26\.465\.2OpenMEVA0\.010580\.00991−\-0\.00067\[−\-0\.00136, 0\.00001\]6\.455\.6

Table B\.22:Median magnitude shareRB​\(d\)=\|B^​d\|/\(\|A^​d\|\+\|B^​d\|\)R\_\{B\}\(d\)=\|\\hat\{B\}d\|/\(\|\\hat\{A\}\\sqrt\{d\}\|\+\|\\hat\{B\}d\|\)of the calibration\-predicted gain over path\-splits \(30% calibration, 50 splits per path\), with the interquartile range atd=0\.10d=0\.10\.RBR\_\{B\}is not netBB, not a share of the realized gain, and not an error reduction\.Table B\.23:Reliability ofB^\\hat\{B\}\. \(a\) Median error terms atd=0\.10d=0\.10by calibration fraction \(50 nested splits per path\)\.AevalA\_\{\\rm eval\}andBevalB\_\{\\rm eval\}are held\-out moment coefficients used only retrospectively, and the sign agreement is moment\-based\. \(b\) Within\-split cluster bootstrap \(30% calibration, 300 draws; clusters are SummEval articles, OpenMEVA stories, and WMT source segments\): median bootstrap SE ofB^\\hat\{B\}, median share of draws keeping the sign ofB^\\hat\{B\}, share of split\-level 95% intervals that include zero, and share of splits on whichAA\-only andA\+BA\{\+\}Bmake different proceed decisions at the threshold0\.1​σ^J0\.1\\,\\hat\{\\sigma\}\_\{J\}\. Panels \(a\) and \(b\) use different split schedules\. Error columns are in native units\.\(a\) Coefficient error by calibration sizeDatasetCal\.d​\|A^−Aeval\|\\sqrt\{d\}\\,\|\\hat\{A\}\-A\_\{\\rm eval\}\|d​\|B^−Beval\|d\\,\|\\hat\{B\}\-B\_\{\\rm eval\}\|RemainderSign agr\. \(%\)SummEval5%0\.03180\.00930\.001778\.410%0\.02260\.00680\.001781\.930%0\.01570\.00440\.001889\.4GPT\-4o5%0\.02140\.00690\.000386\.010%0\.01600\.00490\.000393\.530%0\.01160\.00370\.000497\.0WMT5%0\.16210\.05460\.008279\.510%0\.11870\.04310\.008384\.030%0\.07770\.03170\.008789\.7OpenMEVA5%0\.02380\.00560\.002782\.610%0\.01880\.00440\.002785\.230%0\.01260\.00290\.002790\.0\(b\) Bootstrap sign and interval stabilityDatasetPath\-splitsMedian SE\(B^\)\(\\hat\{B\}\)Sign kept \(%\)CI∋0\\ni 0\(%\)Disagree \(%\)SummEval1,6000\.057096\.752\.60\.0GPT\-4o2000\.0376100\.025\.025\.5WMT7,0500\.387097\.349\.82\.5OpenMEVA5000\.0333100\.035\.20\.0

Table B\.24:Rater count and rating noise on SummEval \(30% calibration\)\. The proxy is a fixed held\-out human rater; calibration gold averageskkof theKKavailable non\-proxy raters, with optional per\-rating noise at nominal scales of the group rating SD, clipped to the observed range; evaluation gold is always the clean mean of all non\-proxy raters\.B^\\hat\{B\}error is the mean\|B^−Beval\|\|\\hat\{B\}\-B\_\{\\rm eval\}\|\. The sign reference differs by block, as each block header states\. The last two columns give the paired CurveMAE difference and its 5,000\-draw path\-cluster bootstrap interval\. Values are not comparable with Table[1](https://arxiv.org/html/2610.41114#S5.T1)b\.CurveMAESettingB^\\hat\{B\}errorSign \(%\)A\+BA\{\+\}BA\+B−AA\{\+\}B\-A\-only95% CIClean ratings,kkofKKgold raters; sign vs\.BevalB\_\{\\rm eval\}Expert, 1 of 20\.061897\.30\.02687−\-0\.01246\[−\-0\.01814,−\-0\.00693\]Expert, 2 of 20\.0428100\.00\.01437−\-0\.01343\[−\-0\.02012,−\-0\.00708\]Crowd, 1 of 40\.065981\.60\.01224−\-0\.00127\[−\-0\.00160,−\-0\.00096\]Crowd, 2 of 40\.061284\.00\.01083−\-0\.00135\[−\-0\.00174,−\-0\.00093\]Crowd, 3 of 40\.059685\.80\.01059−\-0\.00146\[−\-0\.00180,−\-0\.00112\]Crowd, 4 of 40\.059085\.40\.01037−\-0\.00149\[−\-0\.00181,−\-0\.00117\]Noise \(rating SD\), allKKgold raters; sign vs\. cleanB^\\hat\{B\}Expert, Gauss\. 0\.250\.0454100\.00\.01469−\-0\.00865\[−\-0\.01379,−\-0\.00354\]Expert, Gauss\. 0\.500\.054199\.70\.01826−\-0\.00343\[−\-0\.00731, 0\.00031\]Expert, Gauss\. 1\.000\.083599\.50\.032530\.00657\[0\.00465, 0\.00857\]Expert,t3t\_\{3\}0\.250\.044699\.80\.01444−\-0\.00967\[−\-0\.01545,−\-0\.00415\]Expert,t3t\_\{3\}0\.500\.050899\.50\.01662−\-0\.00551\[−\-0\.01011,−\-0\.00113\]Expert,t3t\_\{3\}1\.000\.070999\.30\.025400\.00236\[0\.00041, 0\.00427\]Crowd, Gauss\. 0\.250\.058596\.60\.01082−\-0\.00031\[−\-0\.00068, 0\.00008\]Crowd, Gauss\. 0\.500\.059294\.20\.013040\.00110\[0\.00065, 0\.00163\]Crowd, Gauss\. 1\.000\.058593\.00\.021890\.00335\[0\.00289, 0\.00384\]Crowd,t3t\_\{3\}0\.250\.058196\.90\.01049−\-0\.00062\[−\-0\.00097,−\-0\.00023\]Crowd,t3t\_\{3\}0\.500\.057695\.30\.011860\.00054\[0\.00010, 0\.00101\]Crowd,t3t\_\{3\}1\.000\.056993\.30\.017600\.00263\[0\.00215, 0\.00316\]

Figure B\.7:Sensitivity ofB^\\hat\{B\}to the calibration gold on SummEval\. \(a\) Mean\|B^−Beval\|\|\\hat\{B\}\-B\_\{\\rm eval\}\|against the number of clean gold raters\. \(b\) Sign agreement ofB^\\hat\{B\}with the clean estimate at the same rater count under injected noise \(nominal scale, all gold raters\)\. \(c\) Paired CurveMAE differenceA\+BA\{\+\}BminusAA\-only with 95% path\-cluster intervals: the sign ofB^\\hat\{B\}stays stable, but its predictive benefit disappears at 1\.0 SD for experts and 0\.5 SD for crowd workers\.

### B\.9Open reward\-model and judge proxies

To check that the external\-proxy result is not specific to one judge, we score the same 1,600 SummEval summaries with an open reward model and an open LLM judge and keep the SummEval human ratings as calibration and evaluation targets\. The reward model is Skywork\-Reward\-Llama\-3\.1\-8B\-v0\.2\[[30](https://arxiv.org/html/2610.41114#bib.bib30)\]\(revisiond4117fbf, bfloat16, maximum length 4,096\), which gives one scalar score per summary\. The judge is Qwen2\.5\-32B\-Instruct\[[38](https://arxiv.org/html/2610.41114#bib.bib38)\]\(revision5ede1c97\), which rates each dimension with a rubric prompt that elicits a rationale and a real\-valued score; five passes at temperature 0\.3 are averaged\. The protocol is the prospective protocol of Section[4](https://arxiv.org/html/2610.41114#S4): per dimension, 480 calibration and 1,120 evaluation items, 50 repeats, and endpoint MAE atd=0\.10d=0\.10against the exact held\-out raw\-score tilt gain\.

Table[B\.25](https://arxiv.org/html/2610.41114#A2.T25)shows that addingBBreduces the endpoint MAE ofAA\-only by 27\.1% and 24\.5%\. The calibration two\-term fit is slightly lower thanA\+BA\{\+\}Bfor both proxies and wins about half of the splits; for Qwen, its path\-bootstrap interval relative toA\+BA\{\+\}B,\[−0\.00050,−0\.00003\]\[\-0\.00050,\-0\.00003\], lies entirely below zero\.A\+BA\{\+\}BbeatsAA\-only in 66\.0% and 60\.5% of individual splits, and Qwen coherence is the only one of eight proxy–dimension cells in whichAA\-only is lower \(Table[B\.26](https://arxiv.org/html/2610.41114#A2.T26)\)\. These checks are scoped to SummEval and show that the second\-order benefit is not an artifact of one judge; they do not establish broad proxy generality\. Training\-data overlap between these models and SummEval is not excluded\.

Table B\.25:Open proxies on SummEval: endpoint MAE atd=0\.10d=0\.10, averaged over four dimensions and 50 item splits \(480/1,120\)\. The paired interval forAA\-only minusA\+BA\{\+\}Bis a path\-clustered bootstrap over only four dimension paths \(500 draws\), computed from the saved split rows\.Table B\.26:Open proxies on SummEval by dimension \(endpoint MAE atd=0\.10d=0\.10, 50 splits each\)\.
### B\.10Three reward models: gain forecasts and aggregate errors

This evaluation shows howA^\\hat\{A\}andB^\\hat\{B\}are used for fixed reward\-model proxies; it is not a reward\-model leaderboard\. The proxies are three frozen scalar reward models: Skywork\-Reward\-Llama\-3\.1\-8B\-v0\.2 \(revisiond4117fbf\), ArmoRM\-Llama3\-8B\-v0\.1\[[44](https://arxiv.org/html/2610.41114#bib.bib44)\]\(revisioneb2676d2, final gated scalar\), and OpenAssistant reward\-model\-deberta\-v3\-large\-v2\[[35](https://arxiv.org/html/2610.41114#bib.bib35),[24](https://arxiv.org/html/2610.41114#bib.bib24),[19](https://arxiv.org/html/2610.41114#bib.bib19)\]\(revisionc355404e\)\. There is no fine\-tuning and no new annotation\. The target for each SummEval dimension is the mean of three expert and five crowd ratings\. Every model and summary for an article receive the same article prefix\. The supplied articles are already capped at 2,000 characters, and prefixes are further shortened when needed to fit all three tokenizers \(4,096, 4,096, and 512 tokens\) while keeping each summary in full; the mean retained fraction is 96\.08% of the supplied prefixes\. Predictions use 480 calibration and 1,120 evaluation items, 50 repeats \(seed 0\), and radii0\.0250\.025,0\.050\.05,0\.0750\.075, and0\.100\.10, with splits identical across models\. The splits hold out items, not articles\.

Figure[B\.8](https://arxiv.org/html/2610.41114#A2.F8)summarizes coherence across all 50 matched splits, showing mean gain forecasts and mean within\-split absolute errors separately\. Over all 50 repeats and four dimensions \(Table[B\.27](https://arxiv.org/html/2610.41114#A2.T27)\), addingBBlowers the endpoint MAE in all 12 model–dimension cells and in all 48 cells across the four radii, while the calibration two\-term fit is lower thanA\+BA\{\+\}Bin 7 of 12 endpoint cells\. The repeated splits reuse the same 1,600 items, so split\-to\-split variation should not be interpreted as independent\-sample uncertainty\. Table[B\.28](https://arxiv.org/html/2610.41114#A2.T28)reports full\-pool descriptive coefficients with paired article\-cluster bootstrap intervals\. These differ from the calibration\-only coefficients used for prediction because the estimation sample differs\. The Skywork coherence interval forBBincludes zero, and the other elevenBBintervals are negative\.

The coefficients describe this proxy, target, candidate distribution, and context; they are not an intrinsic quality score of a reward model\. They are invariant to positive affine changes of the proxy but not to rescaling the target or shifting the population, and a negativeBBis a local correction rather than a turning point\. The reliability\-only surrogate is omitted, because its variance ratio depends on proxy scale \(ArmoRM’s scalar reward has standard deviation 0\.022\)\. The article context was preprocessed on this benchmark’s texts before splitting, and training\-data overlap with SummEval is not excluded\. These shared\-context Skywork scores differ from the open\-proxy scores of Appendix[B\.9](https://arxiv.org/html/2610.41114#A2.SS9)\.

Figure B\.8:SummEval coherence forecasts across 50 matched item splits \(480 calibration / 1,120 held\-out items\)\. Top: mean exact held\-out tilt gain \(black\), calibration\-only PACE \(A\+BA\{\+\}B, blue\), andAA\-only forecasts \(orange dashed\)\. Bottom: absolute error computed within each split, then averaged\. Percentages give PACE’s relative MAE change againstAA\-only atd=0\.10d=0\.10\. Shading marks the between\-method gap, not uncertainty: the meanB^​d\\hat\{B\}dcorrection above and the MAE difference below\. Axes are zoomed to showd∈\[0\.025,0\.10\]d\\in\[0\.025,0\.10\]; neither vertical axis starts at zero\. The repeated splits reuse the same 1,600 items\.Table B\.27:Three reward models: endpoint MAE atd=0\.10d=0\.10\(not CurveMAE\), averaged equally over four SummEval dimensions and 50 item splits per dimension \(480/1,120\), in human\-score units\. Point estimates only\. The lower block lists each dimension\.Table B\.28:Full\-pool descriptive coefficients for the three reward models \(all 1,600 items\), with 95% percentile intervals from 1,000 paired article\-cluster bootstrap draws \(seed 1; the same articles are drawn for every model and dimension\)\.BBequals the tail term plus the skewness term\. These are not the calibration\-only coefficients used in Figure[B\.8](https://arxiv.org/html/2610.41114#A2.F8)and Table[B\.27](https://arxiv.org/html/2610.41114#A2.T27), and marginal intervals are not a test of differences between models\.
### B\.11Radius, calibration\-size, and selective\-use robustness checks

This subsection records the final robustness checks used to audit the local diagnostic\. The KL\-radius sweep asks whether the headlined=0\.10d=0\.10result is a single\-radius artifact\. The calibration\-size curve asks how many repeated\-rating calibration items are needed before the coefficient predictor stabilizes\. The residual breakdown checks the finite\-radius approximation error of the local law itself\. The selective GPT\-4o gate summarizes how margin thresholds trade coverage against realized gain in the external\-proxy setting\.

The tables and figures in this subsection summarize the radius sweep, calibration\-size sweep, residual breakdown, and selective GPT\-4o gate\. Table[B\.29](https://arxiv.org/html/2610.41114#A2.T29)gives a compact selection of those results\.

Figure B\.9:KL\-radius audit for the shared local window\. The shaded region and vertical dashed line markd≤0\.10d\\leq 0\.10, the window used for the main prospective prediction tables\. Panel A plots MAE versus KL radius on the human\-proxy setting: theA\+BA\{\+\}Bpredictor remains below theAA\-only and zero baselines throughout the shared local grid\. Panel B gives the same comparison for the GPT\-4o proxy, whereA\+BA\{\+\}Bremains belowAA\-only,AνA\_\{\\nu\}, and zero fromd=0\.005d=0\.005tod=0\.20d=0\.20\. Ribbons or intervals, where shown, are 95% path\-cluster bootstrap intervals for pooled\-split MAE\. Panel C separates split coverage \(the line\) from path counts \(annotations\)\. For SummEval, coverage is relative to 1,589 retained splits: 100% throughd=0\.10d=0\.10,1,472/1,589=92\.6%1\{,\}472/1\{,\}589=92\.6\\%at0\.150\.15, and1,350/1,589=85\.0%1\{,\}350/1\{,\}589=85\.0\\%at0\.200\.20; the corresponding path counts are 32, 31, and 27\. All four GPT\-4o paths remain reachable\.For finite empirical tilt paths, larger KL radii are not always attainable on every split\. We therefore report each radius on the rows that can realize that radius, while retaining lower\-radius rows from the same paths\. This matters mainly for the high\-curvature human\-proxy stress window: all 32 paths are retained throughd=0\.10d=0\.10, while the largerd=0\.20d=0\.20sweep has 27 valid paths and 1,350 valid split rows\.

Table B\.29:Selected robustness\-check results for the radius, calibration\-size, and selective\-use analyses\. “Valid” counts refer to splits or paths that can realize the requested KL radius under the finite empirical exponential\-tilt path\.Tables[B\.30](https://arxiv.org/html/2610.41114#A2.T30)and[B\.31](https://arxiv.org/html/2610.41114#A2.T31)report the full radius sweep and the SummEval/GPT\-4o calibration\-size curves that underlie the selected rows above\. Brackets are 95% path\-cluster bootstrap intervals for pooled\-split MAE\.

Table B\.30:Full KL\-radius sweep atd∈\{0\.005,0\.01,0\.02,0\.05,0\.10,0\.15,0\.20\}d\\in\\\{0\.005,0\.01,0\.02,0\.05,0\.10,0\.15,0\.20\\\}for the human\-proxy SummEval setting and the GPT\-4o external proxy\. Valid path counts at larger radii reflect finite\-empirical tilt\-path reachability\.Table B\.31:Full calibration\-size curve atd=0\.10d=0\.10for the human\-proxy SummEval setting and the GPT\-4o external proxy\. Calibration fractions\{10,20,30,50,70\}%\\\{10,20,30,50,70\\\}\\%are evaluated on the same proxy–gold paths\.The calibration\-size robustness check also repeats the same learning\-curve experiment on the WMT MQM cluster and OpenMEVA\-MAGS ROC/WP slices\. The pattern is consistent with the SummEval result: most of the gain arrives by 30–50% calibration, the coefficient predictor remains below the reliability\-only and zero baselines, and very large calibration fractions can slightly increase variance because the evaluation split becomes smaller\.

Table B\.32:Calibration\-size robustness on additional eligible repeated\-rating analysis slices atd=0\.10d=0\.10\.Reachability filtering at larger radii does not drive the headline result\. Atd=0\.15d=0\.15, the reachable subset differs from the fulld=0\.10d=0\.10population by only−0\.0020\-0\.0020mean gain\. Atd=0\.20d=0\.20, the reachable subset has slightly lower mean gain and less negative calibrationBB, so the stress\-window result should be read as a reachable\-radius audit rather than a full\-population claim\.

Table B\.33:Reachability audit for the high\-radius human\-proxy sweep\. Deltas compare the reachable\-path subset with all 32 paths at the sharedd=0\.10d=0\.10radius\.Because the object is a gain curve rather than a single endpoint, Table[B\.34](https://arxiv.org/html/2610.41114#A2.T34)expands the compact main\-table view with grid\-mean absolute error and maximum error overd∈\{0\.005,0\.01,0\.02,0\.05,0\.10\}d\\in\\\{0\.005,0\.01,0\.02,0\.05,0\.10\\\}, including descriptive empirical\-fit and oracle references\. The ranking is consistent with the endpoint result: the coefficient curve is strong for human\-proxy paths, and on GPT\-4o it remains belowAA\-only,AνA\_\{\\nu\}, and zero on the shared local grid\.

Table B\.34:Expanded curve\-level prediction errors on the shared local gridd≤0\.10d\\leq 0\.10\. Grid MAE averages absolute error overd∈\{0\.005,0\.01,0\.02,0\.05,0\.10\}d\\in\\\{0\.005,0\.01,0\.02,0\.05,0\.10\\\}within each curve, then across curves; this grid differs from Table[1](https://arxiv.org/html/2610.41114#S5.T1)b\. Max\-error columns summarize per\-curve sup\-norm error\.Table[B\.35](https://arxiv.org/html/2610.41114#A2.T35)reports the finite\-radius residual sweep\. TheD≤0\.05D\\leq 0\.05human\-proxy row has median maximum residual0\.00060\.0006for the coefficient predictor, and the sharedD≤0\.10D\\leq 0\.10row remains small relative to the realized gains\. This table separates finite\-radius Taylor error from endpoint\-prediction error\.

Table B\.35:Finite\-radius residual sweep for the local gain curve\.DDis the largest KL radius included in the curve residual computation\. This is a same\-pool approximation audit:theory\_abuses moments of the pool on which the exact curve is computed, not coefficients from a separate calibration split\.aonlydrops curvature,anuis the reliability\-only first\-order approximation, andzeropredicts no gain\. Empirical fits are fitted to this same exact curve\.Table[B\.36](https://arxiv.org/html/2610.41114#A2.T36)and Figure[B\.10](https://arxiv.org/html/2610.41114#A2.F10)audit boundedness and tail sensitivity\. The figure should be read narrowly: Panel A reports only correlation stability of the curvature coefficientBBrelative to the original preprocessing, notAA\-stability or sign agreement, and Panel B reports the median absolute change in exact endpoint gain atd=0\.10d=0\.10relative to the original preprocessing rather than the prediction error of the local law\. Table[B\.36](https://arxiv.org/html/2610.41114#A2.T36)therefore separatesAA\-stability,BB\-correlation,BB\-sign agreement, gain\-sign agreement, gain\-shift size, and MAE\. Mild winsorization and controlled gold\-side outliers are comparatively benign in this audit, rank\-normalization can substantially change endpoint gain scale, especially in WMT, and proxy\-side outliers can strongly perturb curvature, especially in OpenMEVA\. We therefore interpret plug\-inBBas a useful local diagnostic rather than a tail\-robust guarantee\.

Table B\.36:Tail and outlier robustness audit\. The columns separateAA\-stability, curvature\-coefficient correlation,BB\-sign agreement, gain\-sign agreement, median absolute endpoint\-gain shift atd=0\.10d=0\.10relative to the original preprocessing, and within\-condition MAE\. Mild winsorization and controlled gold outliers are comparatively benign, rank\-normalization can change endpoint gain scale, and proxy outliers can strongly perturb curvature\.Figure B\.10:Tail and outlier robustness audit\. Panel A reports stability of the curvature coefficientBB, measured as correlation with the original\-preprocessing coefficient\. Panel B reports the median absolute change in exact endpoint gain atd=0\.10d=0\.10relative to the original preprocessing; this is a gain\-scale shift, not the prediction error of the local law\. Mild winsorization and controlled gold\-side outliers are comparatively benign in this audit\. Rank\-normalization can change endpoint gain scale, especially for WMT, and proxy\-side outliers can substantially perturb curvature, especially in OpenMEVA\. Sign\-agreement andAA\-stability metrics are reported in Table[B\.36](https://arxiv.org/html/2610.41114#A2.T36)\.Figure B\.11:Radius and calibration\-size sensitivity: panels A/C are SummEval human\-proxy paths; B/D are GPT\-4o proxy paths\. Panels A and B report MAE as the KL radiusddincreases; the shaded region and dashed line mark the sharedd≤0\.10d\\leq 0\.10local window and main endpoint\. Panels C and D report MAE atd=0\.10d=0\.10as the calibration split fraction varies on the same proxy–gold paths; larger calibration fractions leave smaller evaluation splits\. Ribbons or intervals, where shown, are 95% path\-cluster bootstrap intervals for pooled\-split MAE\. The shared legend compares coefficientA\+BA\{\+\}B,AA\-only, the empirical fit, reliability\-onlyAνA\_\{\\nu\}, and zero gain\.Figure B\.12:Finite\-range residual and selective\-use diagnostics\. Panels A/B are same\-pool approximation audits for SummEval/GPT\-4o: moments and empirical fits use the same pool and exact curve whose median maximum residual is reported overd≤Dd\\leq D\. They do not measure held\-out prediction error\. Panel C is the GPT\-4o external\-proxy selective\-use audit:τ\\tauthresholds the predicted endpoint\-gain margin before applying the optimize\-if\-positive rule, and lower\-margin cases are abstained rather than treated as confident positive selections\.
### B\.12OOD transfer details

The OOD experiments use calibration\-only target standardization \(z\_cal\), a 30% target calibration split, and 50 random repeats\. We report SummEval→\\rightarrowWMT MQM clusters, WMT MQM clusters→\\rightarrowSummEval, and SummEval→\\rightarrowOpenMEVA\-MAGS ROC/WP\. Forθ=\(A,B\)\\theta=\(A,B\), the source\-prior predictor is

θ^shrink=λ​θ^target\+\(1−λ\)​θ¯source,\\hat\{\\theta\}\_\{\\rm shrink\}=\\lambda\\hat\{\\theta\}\_\{\\rm target\}\+\(1\-\\lambda\)\\bar\{\\theta\}\_\{\\rm source\},whereθ¯source\\bar\{\\theta\}\_\{\\rm source\}is the mean source coefficient vector\. The main table usesλ=0\.75\\lambda=0\.75as a fixed target\-heavy shrinkage value for the summary row, not a value tuned on evaluation targets\. Table[B\.39](https://arxiv.org/html/2610.41114#A2.T39)reports the fullλ∈\{0,\.25,\.5,\.75,1\}\\lambda\\in\\\{0,\.25,\.5,\.75,1\\\}sweep\. The earlier SummEval→\\rightarrowWMT\-clusterz\_bothvariant, which uses the full target split for standardization, gives similar conclusions, butz\_calis the more conservative prospective protocol\.

Table B\.37:Cross\-slice transfer prediction from SummEval to eligible WMT MQM clusters underz\_cal\. TheAνA\_\{\\nu\}row is the target\-side reliability\-only baseline\. MAE intervals atd=0\.10d=0\.10are percentile intervals from 2,000 path\-level bootstrap draws \(seed 0\), averaging the 50 splits within each WMT target path\.Table[B\.38](https://arxiv.org/html/2610.41114#A2.T38)summarizes all three routes atd=0\.10d=0\.10\. SummEval→\\rightarrowWMT clusters tests whether a summarization prior helps an eligible translation\-quality slice, WMT→\\rightarrowSummEval tests the reverse direction, and SummEval→\\rightarrowOpenMEVA\-MAGS tests transfer into a different generation domain on a smaller slice\. Source\-only transfer beats zero gain on all three routes andAνA\_\{\\nu\}on two of three, so it is informative but not invariant\. Target calibration improves on source\-only transfer on every route\. The fixed target\-heavy source prior \(λ=0\.75\\lambda=0\.75, not tuned on evaluation targets\) is slightly below target\-only calibration on both WMT routes but above it on SummEval→\\rightarrowOpenMEVA, so source coefficients are useful priors that target calibration should update before the forecast is used\.

Table B\.38:Cross\-slice transfer atd=0\.10d=0\.10under calibration\-only target standardization \(z\_cal\)\. WMT routes use the eligible MQM cluster slice, and OpenMEVA uses MAGS ROC/WP\. Source\-prior uses a fixedλ=0\.75\\lambda=0\.75target\-heavy shrinkage value\. Appendix[B\.12](https://arxiv.org/html/2610.41114#A2.SS12)gives interval and sensitivity details where available\. Lower MAE is better\.The most important OOD metric is magnitude MAE, not sign accuracy\. The calibration\-only protocol confirms the main conclusion across directions: target calibration reduces error relative to source\-only transfer on all three routes, but fixed source\-prior shrinkage does not consistently improve over target\-only calibration\. The corresponding SummEval→\\rightarrowWMT\-clusterz\_bothMAE values atd=0\.10d=0\.10are0\.04070\.0407for source mean,0\.02580\.0258for target calibration,0\.02350\.0235for source prior,0\.20530\.2053for targetAνA\_\{\\nu\}, and0\.24320\.2432for zero gain\.

Table B\.39:OOD transfer sensitivity to source\-prior shrinkage atd=0\.10d=0\.10\.λ=0\\lambda=0is source\-only andλ=1\\lambda=1is target calibration only\.
### B\.13GPT\-4o external\-proxy diagnostics

The available GPT\-4o proxy file covers all 1,600 SummEval items\. The GPT\-4o external\-proxy experiment differs from the human\-proxy setting primarily in proxy ranking and top\-tail alignment\. Table[B\.17](https://arxiv.org/html/2610.41114#A2.T17)reports the current margin\-conditioned fixed\-gain sign behavior, and Table[B\.40](https://arxiv.org/html/2610.41114#A2.T40)summarizes the broader rank and tail diagnostics\. The median proxy standard deviation is0\.6780\.678, and the median model/human proxy\-standard\-deviation ratio across dimensions is0\.9090\.909\. Median Spearman is0\.2850\.285, median Pearson is0\.3290\.329, and median top\-10% overlap with the human\-gold top set is0\.1500\.150\. Relative to the human\-proxy view, the median drops are0\.2880\.288in Spearman and0\.2560\.256in top\-10% overlap\.

Table B\.40:GPT\-4o external\-proxy ranking and tail diagnostics\. Overall rows are medians over SummEval dimensions unless otherwise stated\.
### B\.14External\-judge reproducibility record

External judges can change across model snapshots and prompts, so the GPT\-4o result is treated as an external\-proxy evaluation of one fixed model\-and\-prompt configuration rather than a claim about all LLM judges\. The reproducibility package includes the aggregate proxy scores and the scripts used for the reported analysis\. Table[B\.41](https://arxiv.org/html/2610.41114#A2.T41)summarizes what can be recovered from the available records and which fields are not recoverable from the aggregate file alone\. In particular, the aggregate artifact is sufficient to reproduce the reported post\-processing analyses, but exact API reruns are not fully reproducible without a raw per\-call log containing the exact model snapshot, decoding parameters, and invalid\-output/retry counts\.

Table B\.41:Reproducibility record for the GPT\-4o external\-proxy configuration\.
### B\.15Finite\-range diagnostics and best\-of\-NNbridge

Finite\-range diagnostics check whether thed≤0\.1d\\leq 0\.1window used for local fitting is numerically regular\. The SummEval human\-proxy setting has median minimum tilted variance0\.6310\.631and median minimum effective sample size 1338\. GPT\-4o proxy has median minimum variance0\.3750\.375and ESS 1327\. Thus the GPT\-4o external\-proxy behavior is not caused by tilt\-path collapse on the shared local range\. Its tail quality is less favorable than the human\-proxy setting: the median tail\-at\-max minus baseline is−0\.054\-0\.054, with 50% of paths below baseline\. Table[B\.42](https://arxiv.org/html/2610.41114#A2.T42)and Figure[B\.15](https://arxiv.org/html/2610.41114#A2.F15)report the finite\-range audit, while Tables[B\.43](https://arxiv.org/html/2610.41114#A2.T43)–[B\.46](https://arxiv.org/html/2610.41114#A2.T46)and Figure[B\.14](https://arxiv.org/html/2610.41114#A2.F14)report the Best\-of\-NNbridge\.

Table B\.42:Finite\-range diagnostics ond≤0\.1d\\leq 0\.1\. Cumulant columns are medians across paths of each path’s maximum absolute cumulant\.Best\-of\-NNis a discrete selection rule, not the smooth exponential\-tilt path analyzed by the theorem\. Table[B\.43](https://arxiv.org/html/2610.41114#A2.T43)computes the exact pooled BoN distribution, with replacement and uniform tie\-breaking, on the full item set of each of 191 paths and forN∈\{2,3,4,6,8,12,16,24,32,48,64\}N\\in\\\{2,3,4,6,8,12,16,24,32,48,64\\\}\. AtN=2N=2, the median induced radius is0\.1660\.166–0\.1930\.193, outside the checked window in every setting, and the near\-untied ROUGE\-1 proxy matches the atomless closed formsdN=log⁡N−\(N−1\)/Nd\_\{N\}=\\log N\-\(N\-1\)/NandRelESSN=\(2​N−1\)/N2\\mathrm\{RelESS\}\_\{N\}=\(2N\-1\)/N^\{2\}\. Only 4 of the 2,101 nontrivial points \(0\.19%\) lie insided≤0\.10d\\leq 0\.10\. All four are SummEval held\-out expert paths atN=2N=2\(three consistency raters and one fluency rater; Table[B\.44](https://arxiv.org/html/2610.41114#A2.T44)\), on which 85\.4–89\.1% of the 1,600 items carry the top proxy score\. BoN and the pooled tilt weight items only through their proxy score, so both concentrate on that tied block, and the reachable KL is capped atlog⁡\(m/ntop\)=0\.116\\log\(m/n\_\{\\rm top\}\)=0\.116–0\.1570\.157\. At these four points, the coefficients and the descriptive fit are computed on the same pool on which BoN is evaluated, so the comparison is a same\-pool audit rather than a held\-out prediction\. All listed nonzero predictors overestimate the four BoN gains; zero underestimates them; the mean absolute error is0\.01970\.0197forA\+BA\{\+\}B,0\.08840\.0884forAA\-only,0\.12380\.1238forAνA\_\{\\nu\},0\.24900\.2490for zero gain, and0\.00430\.0043for the descriptive two\-term fit\. These are scope exceptions from one dataset, not evidence that BoN generally lands in the window\.

Table[B\.45](https://arxiv.org/html/2610.41114#A2.T45)sweeps the window continuously with two softened rules on 30/70 splits \(coefficients from the calibration split, errors on the evaluation split\): the mixturepα=\(1−α\)​p0\+α​pBoN,8p\_\{\\alpha\}=\(1\-\\alpha\)p\_\{0\}\+\\alpha p\_\{\{\\rm BoN\},8\}and the rank tiltqλ,i∝p0,i​Uiλq\_\{\\lambda,i\}\\propto p\_\{0,i\}U\_\{i\}^\{\\lambda\}, whereUiU\_\{i\}is the one\-based average rank ofrir\_\{i\}divided by the pool size \(ties receive their average rank\)\. Inside0\.005<d≤0\.100\.005<d\\leq 0\.10,A\+BA\{\+\}Bis below zero gain andAνA\_\{\\nu\}in every cell and within0\.0010\.001of the calibration two\-term fit\. It improves onAA\-only for SummEval and WMT, favors it for the GPT\-4o mixture, nearly ties it for the GPT\-4o rank tilt, and is worse thanAA\-only on OpenMEVA\. This supports an informative local approximation for smooth rules inside the window, not universal dominance of the curvature term\. Only two to four grid values per split fall inside the window, mostly in its lower half\.

The prompt\-conditional audit in Table[B\.46](https://arxiv.org/html/2610.41114#A2.T46)and Figure[B\.13](https://arxiv.org/html/2610.41114#A2.F13)applies BoN within each prompt instead of over the pooled item set\. Even atN=2N=2, the mean prompt KL is outside the local window \(0\.144 on SummEval and 0\.131 on OpenMEVA\), and only 12\.5% and 11\.7% of prompts are inside\. AtN=4N=4, only the six saturated SummEval expert paths, on which 78–89% of items share the top score, remain reachable by a pooled raw\-score tilt withind≤0\.25d\\leq 0\.25; forN≥8N\\geq 8, no path is\. ForN≥4N\\geq 4, the prompts with prompt KL at most 0\.10 \(5\.8% on SummEval and 3\.0% on OpenMEVA\) are prompts whose candidates are all or almost all tied, not instances of mild selection\. This mismatch is why BoN is reported as a scope audit rather than as another validation of the smooth local theorem\.

Table B\.43:Exact pooled Best\-of\-NN\(with replacement, uniform tie\-breaking\) over each path’s full item set;dBoN=KL\(p∥p0\)d\_\{\\rm BoN\}=\\operatorname\{KL\}\(p\\\|p\_\{0\}\)in nats andRelESS=ESS/m\\mathrm\{RelESS\}=\\mathrm\{ESS\}/m\. Nontrivial meansN≥2N\\geq 2\(11 values per path\); counts are \(path,NN\) points with their share\. Held\-out\-human rows use a held\-out rater as the proxy; the ROUGE\-1 and GPT\-4o rows use that model\. Item pools havem=1,600m=1\{,\}600\(SummEval\),430430–2,8202\{,\}820\(WMT\), and1,0001\{,\}000\(OpenMEVA\) items\. MAE uses only the four in\-window points \(same\-pool coefficients,n=4n=4\); – means no in\-window point\. Thed≤0\.25d\\leq 0\.25counts include saturated paths whose reachable KLlog⁡\(m/ntop\)\\log\(m/n\_\{\\rm top\}\)is below 0\.25\. The lower panel gives medians byNN\.Median atN=2N=2Nontrivial points \(N≥2N\\geq 2\)In\-window MAESettingPathsdBoNd\_\{\\mathrm\{BoN\}\}RelESSd≤0\.10d\\leq 0\.10d≤0\.25d\\leq 0\.25A\+BA\-onlySummEval human320\.1720\.7724 \(1\.1%\)92 \(26\.1%\)0\.01970\.0884SummEval ROUGE\-140\.1930\.7500 \(0\.0%\)4 \(9\.1%\)––SummEval GPT\-4o40\.1890\.7540 \(0\.0%\)4 \(9\.1%\)––WMT MQM1410\.1860\.7600 \(0\.0%\)154 \(9\.9%\)––OpenMEVA\-MAGS100\.1660\.7710 \(0\.0%\)10 \(9\.1%\)––All settings1910\.1850\.7614 \(0\.2%\)264 \(12\.6%\)0\.01970\.0884SettingN=2N=2N=3N=3N=4N=4N=8N=8N=16N=16N=64N=64MediandBoNd\_\{\\mathrm\{BoN\}\}\(nats\)SummEval human0\.1720\.3770\.5490\.9881\.2331\.266SummEval ROUGE\-10\.1930\.4320\.6361\.2041\.8353\.174SummEval GPT\-4o0\.1890\.4210\.6191\.1711\.7683\.000WMT MQM0\.1860\.4060\.5820\.9591\.1171\.131OpenMEVA\-MAGS0\.1660\.3790\.5601\.0591\.5922\.563Median relative ESSSummEval human0\.7720\.5920\.4800\.3250\.2850\.282SummEval ROUGE\-10\.7500\.5560\.4380\.2340\.1210\.031SummEval GPT\-4o0\.7540\.5620\.4440\.2420\.1280\.034WMT MQM0\.7600\.5830\.4840\.3520\.3240\.323OpenMEVA\-MAGS0\.7710\.5840\.4680\.2650\.1460\.051

Table B\.44:The four nontrivial BoN points insided≤0\.10d\\leq 0\.10, all SummEval held\-out expert paths atN=2N=2\(m=1,600m=1\{,\}600; gold is the mean of the other two experts\)\. Coefficients and the empirical fit use the same full pool on which BoN is evaluated \(descriptive, not held\-out\)\. Gains, predictions, and errors are in SummEval 1–5 units\.Table B\.45:Softened BoN\-direction rules inside0\.005<d≤0\.100\.005<d\\leq 0\.10: the mixture with BoN atN=8N=8and the rank tilt\. Pooled item\-set rules \(not prompt\-conditional\); coefficients from 30% calibration splits, errors on the 70% evaluation items, 30 splits per path\. Path\-macro MAE weights each path equally; “Paths: A\+B wins” counts paths on whichA\+BA\{\+\}Bhas lower error thanAA\-only\. Only two to four grid values per split fall in the window\. Native units, not comparable across datasets\.Table B\.46:Prompt\-conditional Best\-of\-NNscope audit\. KL is computed within each prompt\. The reported mean averages path\-level prompt means; the median is the median of path\-level prompt medians\. “Pooled tilt reachable” is the share of paths whose mean prompt KL is below both 0\.25 and the pooled tilt’s supremumlog⁡\(m/ntop\)\\log\(m/n\_\{\\rm top\}\), so that one pooled raw\-score tilt attains it\. EvenN=2N=2is outside the local window on average and for most prompts\. AtN=4N=4, only the six saturated SummEval expert paths remain reachable; forN≥8N\\geq 8, no path is\. ForN≥4N\\geq 4, the prompts with KL at most 0\.10 are prompts whose candidates are \(almost\) all tied\.Figure B\.13:Prompt\-conditional Best\-of\-NNscope audit\. Panel A reports the mean prompt\-level KL of exact with\-replacement prompt\-conditional BoN, with thed=0\.10d=0\.10window shown by the dashed line\. Panel B reports the fraction of paths whose mean prompt KL is attainable by a pooled raw\-score tilt withind≤0\.25d\\leq 0\.25, accounting for paths whose tied top scores cap the pooled tilt\. Mean BoN gains are in Table[B\.46](https://arxiv.org/html/2610.41114#A2.T46)\.![Refer to caption](https://arxiv.org/html/2610.41114v1/fig_bon_bridge_bootstrap.png)Figure B\.14:Legacy pooled\-item BoN resampling audit, not prompt\-conditional selection or a theorem test\. Candidates are drawn with replacement from the pooled item set, and each candidate’s proxy is an independently drawn rating or RewardBench model column; gold is the itemwise mean over all columns, not LOO\. Panels A–D use the bridge coordinateu=log⁡Nu=\\log N, not the induced KL\. Panels B–C compare fitted and theory coefficients, and Panel D shows fittedBBas the fit range grows\. Panels E/F show the proxy–gold gap/gold level, not gold gain\. The displayedν\\nuis a square\-root variance ratio, not ICC\.Figure B\.15:Finite\-range diagnostics across eligible analysis slices\. Panel A reports median minimum tilted variance on the left scale and median minimum effective sample size on the accompanying labels/right scale for each slice on the shared local windowd≤0\.1d\\leq 0\.1; tail labels summarize the share of paths whose tilted tail gold falls below the reference baseline\. Panel B relates median minimum tilted variance to curvature\-sign agreement with the evaluation curve\-fit coefficient\. This is a numerical stability diagnostic, not a guarantee of accurate prediction\.
### B\.16LoRA\-DPO bridge details

#### Setup\.

The reference policy is Qwen2\.5\-3B\-Instruct\. From 1,000 UltraFeedback prompts\[[9](https://arxiv.org/html/2610.41114#bib.bib9)\]\(pinned revision\), the reference policy generates eight candidates per prompt; the highest\- and lowest\-scoring candidates under Skywork\-Reward\-Llama\-3\.1\-8B\-v0\.2, the optimized proxyrr, form 968 preference pairs with a nonzero score gap\. OpenAssistant reward\-model\-deberta\-v3\-large\-v2\[[35](https://arxiv.org/html/2610.41114#bib.bib35)\]supplies the frozen silver targetJJand is never used for training\. DPO usesβ=0\.1\\beta=0\.1, learning rate10−510^\{\-5\}, effective batch size 16, and LoRA rank, alpha, and dropout of 16, 32, and 0\.05; the three seeds share data and pools and differ only in training randomness, with separate training runs of 64, 90, 120, 150, 180, and 200 steps, each giving one checkpoint\. The training, calibration \(100 prompts\), and evaluation \(200 prompts\) prompt sets are disjoint\. Calibration and evaluation pools are fixed, hash\-verified sets of 32 candidates per prompt from the reference policy \(temperature 0\.9, top\-pp0\.95, at most 320 new tokens\)\. One calibration prompt has zero variance in both proxy and silver scores and is excluded\. For each of the 99 remaining prompts,\(A^x,B^x\)\(\\hat\{A\}\_\{x\},\\hat\{B\}\_\{x\}\)is computed on its 32 candidates and averaged uniformly, givingA^=0\.2268\\hat\{A\}=0\.2268andB^=−0\.0465\\hat\{B\}=\-0\.0465\. The surrogateAνA\_\{\\nu\}is2​Var^​\(J\)/SD^​\(r\)\\sqrt\{2\}\\,\\widehat\{\\operatorname\{Var\}\}\(J\)/\\widehat\{\\mathrm\{SD\}\}\(r\)per prompt, averaged \(0\.25990\.2599\); no rater reliability exists for a deterministic silver target\.

#### Estimand\.

For evaluation promptxx,wx,i∝exp⁡\{log⁡pDPO​\(yi∣x\)−log⁡pref​\(yi∣x\)\}w\_\{x,i\}\\propto\\exp\\\{\\log p\_\{\\rm DPO\}\(y\_\{i\}\\mid x\)\-\\log p\_\{\\rm ref\}\(y\_\{i\}\\mid x\)\\\}is normalized over its 32 candidates, anddx=KL\(wx∥Unif32\)d\_\{x\}=\\operatorname\{KL\}\(w\_\{x\}\\\|\\mathrm\{Unif\}\_\{32\}\)\. This KL is defined directly between two discrete distributions and needs no tilt assumption; the tilt enters only through the idealized comparator, which reweights the same pool by proxy score to matchdxd\_\{x\}\. The realized gain is thewxw\_\{x\}\-weighted silver mean minus the uniform\-pool mean\. Normalized ratios are not the DPO policy restricted to the pool, and because the candidates came from a temperature and top\-ppproposal while the ratios use untempered log\-probabilities, they are not proposal\-corrected importance weights\. The realized gain is therefore a self\-normalized diagnostic, not a full\-policy or fresh\-generation estimate\. Predictions averageA^​dx\+B^​dx\\hat\{A\}\\sqrt\{d\_\{x\}\}\+\\hat\{B\}d\_\{x\}over evaluation prompts, and the error of a checkpoint is the absolute difference between the prompt\-averaged prediction and the prompt\-averaged target\.

#### Checkpoint gate and radius tail\.

We keep checkpoints whose mean prompt KL is at most 0\.10: steps 64, 90, 120, and 150 for seed 0 and steps 64, 90, and 120 for seeds 1 and 2\. A median\-KL gate keeps the same ten checkpoints\. The gate uses no silver label, gain, or error, but it is transductive in the reported radius\. Not every prompt is local: across the 2,000 retained prompt–checkpoint cells, the median, 90th, 95th, and 99th percentiles and the maximum ofdxd\_\{x\}are 0\.0257, 0\.0919, 0\.1224, 0\.1867, and 0\.4668, and 8\.7% exceed 0\.10\. Errors are averaged over the retained checkpoints within each seed, and Table[3](https://arxiv.org/html/2610.41114#S5.T3)reports the mean and sample SD across the three seed averages\.AA\-only minusA\+BA\{\+\}Brealized MAE is 0\.00187, 0\.00177, and 0\.00189 for seeds 0, 1, and 2\. At all ten checkpoints, the ordering is realized gain<<A\+BA\{\+\}Bprediction<<AA\-only prediction<<idealized gain\. Both forecasts therefore underpredict the frontier, and the negativeBBmovesA\+BA\{\+\}Btoward the realized gain\. DPO changes a conditional distribution per prompt, whereas the bridge averages prompt\-specific coefficients into one pair, so it is an empirical conditional\-optimization bridge rather than an in\-window theorem validation\.

#### Comparators\.

Table[B\.47](https://arxiv.org/html/2610.41114#A2.T47)reports 46 predictor variants against both targets, grouped by information setting\. Fourteen have lower realized\-gain MAE thanA\+BA\{\+\}B, including a Spearman\-scaledAA\-only variant \(0\.013780\.01378\) and pre\-pool flexible calibrators on a within\-prompt rank coordinate \(0\.010960\.01096for binned\)\. For the same fittedh^\\hat\{h\}, the aggregate projection–direct gap is at most0\.000370\.00037\. Realized\-gain MAE mixes predictor error with the optimizer’s shortfall, whose mean is0\.021160\.02116, so a lower realized MAE can arise from cancellation; frontier MAE and realized MAE answer different questions, and no variant is universally best\. The four cross\-validated pre\-pool rows \(binned and spline on raw or prompt\-rank proxies\) depend on the tie order of grouped cross\-validation among 100 equal\-size prompt groups, and a rerun on different hardware selects different hyperparameters; we report the original outputs\. The other cross\-validated binned and spline rows use the same scheme and could shift likewise\.

Table B\.47:All DPO comparators on the ten mean\-KL\-gated checkpoints of Table[3](https://arxiv.org/html/2610.41114#S5.T3)\(mean±\\pmsample SD over three seed means; lower is better\)\. The realized gain is a self\-normalized likelihood\-ratio reweighting of fixed pools, scored by a frozen silver reward model; it is not a fresh\-generation, human\-rated, or exact full\-policy gain\. \(a\) Pre\-pool: calibration prompts and measureddxd\_\{x\}only\. \(b\) Target\-pool\-aware direct integration:h^\\hat\{h\}fitted on calibration prompts and integrated over each evaluation prompt’s proxy scores\. \(c\) Projection of the sameh^\\hat\{h\}; the last column is its projection\-minus\-direct prediction gap\. \(d\) Cross\-fitted diagnostics that use evaluation silver labels; they are not deployable\. MLP rows are exploratory\.MAE vs idealizedMAE vs realizedProjection−\-directPredictortilt frontierlikelihood\-ratio gaingap\(a\) Pre\-pool: calibration prompts and measureddxd\_\{x\}onlyA\+BA\{\+\}B0\.00395±\\pm0\.000470\.01721±\\pm0\.00116–AA\-only0\.00210±\\pm0\.000530\.01905±\\pm0\.00120–SpearmanAA\-only variant0\.00738±\\pm0\.000450\.01378±\\pm0\.00113–AνA\_\{\\nu\}0\.00371±\\pm0\.000610\.02487±\\pm0\.00129–Zero0\.04197±\\pm0\.000170\.02081±\\pm0\.00062–Calibration empirical curve0\.00430±\\pm0\.000470\.01686±\\pm0\.00116–Raw isotonic \(frozen curve\)0\.01878±\\pm0\.000800\.03994±\\pm0\.00150–Raw binned \(frozen curve\)0\.00193±\\pm0\.000580\.02309±\\pm0\.00126–Raw spline \(frozen curve\)0\.00131±\\pm0\.000550\.02247±\\pm0\.00124–Raw MLP \(frozen curve\)0\.01497±\\pm0\.000750\.03612±\\pm0\.00144–Prompt\-rank isotonic \(frozen curve\)0\.00433±\\pm0\.000470\.01683±\\pm0\.00116–Prompt\-rank binned \(frozen curve\)0\.01020±\\pm0\.000390\.01096±\\pm0\.00107–Prompt\-rank spline \(frozen curve\)0\.00804±\\pm0\.000420\.01312±\\pm0\.00111–Prompt\-rank MLP \(frozen curve\)0\.00577±\\pm0\.000450\.01539±\\pm0\.00114–\(b\) Target\-pool\-aware: direct integration ofh^​\(r\)\\hat\{h\}\(r\)Raw isotonic0\.01786±\\pm0\.000670\.03902±\\pm0\.00138–Raw binned0\.00237±\\pm0\.000550\.02353±\\pm0\.00127–Raw spline0\.00239±\\pm0\.000450\.02355±\\pm0\.00116–Prompt\-centered isotonic0\.00059±\\pm0\.000390\.02170±\\pm0\.00112–Prompt\-centered binned0\.00573±\\pm0\.000400\.01543±\\pm0\.00108–Prompt\-centered spline0\.00274±\\pm0\.000430\.01842±\\pm0\.00111–Prompt\-zzisotonic0\.00088±\\pm0\.000500\.02031±\\pm0\.00116–Prompt\-zzbinned0\.01044±\\pm0\.000410\.01072±\\pm0\.00110–Prompt\-zzspline0\.00377±\\pm0\.000480\.01739±\\pm0\.00116–Prompt\-rank isotonic0\.00385±\\pm0\.000470\.01731±\\pm0\.00116–Prompt\-rank binned0\.00903±\\pm0\.000410\.01213±\\pm0\.00110–Prompt\-rank spline0\.00624±\\pm0\.000440\.01492±\\pm0\.00113–\(c\) Target\-pool\-aware: two\-term projection of the sameh^​\(r\)\\hat\{h\}\(r\)Raw isotonic0\.01813±\\pm0\.000670\.03929±\\pm0\.001380\.00027±\\pm0\.00002Raw binned0\.00262±\\pm0\.000560\.02378±\\pm0\.001280\.00025±\\pm0\.00001Raw spline0\.00257±\\pm0\.000440\.02372±\\pm0\.001150\.00017±\\pm0\.00001Prompt\-centered isotonic0\.00063±\\pm0\.000450\.02179±\\pm0\.001120\.00008±\\pm0\.00001Prompt\-centered binned0\.00559±\\pm0\.000410\.01557±\\pm0\.001080\.00014±\\pm0\.00001Prompt\-centered spline0\.00264±\\pm0\.000440\.01852±\\pm0\.001110\.00010±\\pm0\.00001Prompt\-zzisotonic0\.00085±\\pm0\.000500\.02033±\\pm0\.001160\.00002±\\pm0\.00000Prompt\-zzbinned0\.01007±\\pm0\.000430\.01109±\\pm0\.001110\.00037±\\pm0\.00003Prompt\-zzspline0\.00370±\\pm0\.000480\.01746±\\pm0\.001160\.00007±\\pm0\.00001Prompt\-rank isotonic0\.00381±\\pm0\.000470\.01735±\\pm0\.001160\.00004±\\pm0\.00000Prompt\-rank binned0\.00878±\\pm0\.000420\.01238±\\pm0\.001110\.00024±\\pm0\.00002Prompt\-rank spline0\.00614±\\pm0\.000440\.01502±\\pm0\.001130\.00010±\\pm0\.00001\(d\) Evaluation\-prompt cross\-fit diagnostic \(uses evaluation silver labels\)Raw isotonic \(cross\-fit\)0\.00809±\\pm0\.000340\.02924±\\pm0\.00102–Raw spline \(cross\-fit\)0\.00185±\\pm0\.000300\.02290±\\pm0\.00109–Prompt\-centered isotonic \(cross\-fit\)0\.00514±\\pm0\.000500\.02630±\\pm0\.00114–Prompt\-centered spline \(cross\-fit\)0\.00347±\\pm0\.000480\.02463±\\pm0\.00114–Prompt\-zzisotonic \(cross\-fit\)0\.00201±\\pm0\.000540\.02317±\\pm0\.00120–Prompt\-zzspline \(cross\-fit\)0\.00126±\\pm0\.000510\.01990±\\pm0\.00119–Prompt\-rank isotonic \(cross\-fit\)0\.00197±\\pm0\.000500\.01919±\\pm0\.00118–Prompt\-rank spline \(cross\-fit\)0\.00383±\\pm0\.000480\.01733±\\pm0\.00117–

### B\.17Measurement intervention

The intervention experiment aggregates more raters into the proxy while holding the item set and target notion fixed\. The result is intentionally mixed\. Reliability andAAimprove almost uniformly, but surrogate headroom depends on both signal and curvature\. The reported headroom statistic is the vertex height of the concave two\-term surrogate:

H∗​\(A,B\)=−A24​B,A\>0,B<0\.H\_\{\*\}\(A,B\)=\-\\frac\{A^\{2\}\}\{4B\},\\qquad A\>0,\\ B<0\.It is recorded as missing outside this positive\-alignment, negative\-curvature regime\. The vertex can lie outside the checked KL window, so this is an extrapolated surrogate statistic, not a validated maximum attainable gain or a maximum restricted tod≤Dd\\leq D\. The headroom\-error summary in Table[B\.12](https://arxiv.org/html/2610.41114#A2.T12)uses the same convention\. In SummEval, moving fromk=1k=1tok=2k=2raisesAAin all four groups but makesBBmore negative and decreases surrogate headroom in all four\. The eligible WMT MQM cluster slice has positive median headroom change, but only 12\.8% of groups have a positive 2\.5th percentile across random rater partitions, not a bootstrap confidence bound\.

Table B\.48:Measurement\-intervention summary\.H∗H\_\{\*\}is the extrapolated surrogate vertex height−A2/\(4B\)\-A^\{2\}/\(4B\)forA\>0,B<0A\>0,B<0, not a checked\-window maximum\.Δ​H\\Delta Hcombines changes in signal and curvature;q\.025q\_\{\.025\}is its empirical 2\.5th percentile over random rater partitions\.Figure B\.16:Protocol split\-half stability\. Panel A reports coherence split\-half correlations under mean, median, and trimmed\-mean proxy construction\. Panel B gives the same comparison for consistency\. Panel C gives the same comparison for fluency\. Panel D gives the same comparison for relevance\. Error bars summarize random rater\-half splits on the same items\. The panels describe sensitivity to rater composition, while Table[B\.48](https://arxiv.org/html/2610.41114#A2.T48)shows that improving measurement changes both signal and curvature\.
### B\.18Proxy perturbation baselines

Proxy perturbations isolate whether the exact coefficient law is sensitive to actual proxy–gold alignment\. The main text summarizes this sanity check\. Here Table[B\.49](https://arxiv.org/html/2610.41114#A2.T49)gives the full noise sweep, Figure[B\.17](https://arxiv.org/html/2610.41114#A2.F17)shows the actual\-gain bars for the single\-slice stress test, and Figure[B\.18](https://arxiv.org/html/2610.41114#A2.F18)repeats the audit across eligible slices\. Oracle and original proxies produce positive gains and positive exactAA\. The random\-normal and shuffle controls answer slightly different questions\. A random\-normal proxy replaces the original score with independent noise, so both the induced ranking and the score distribution change\. A shuffled proxy preserves the original proxy histogram and variance but destroys itemwise proxy–gold alignment\. In both controls, the realized gain and exactAAare close to zero on average, showing that the coefficient law is tracking directional alignment rather than proxy scale alone\. The shuffle result is especially diagnostic becauseAνA\_\{\\nu\}stays spuriously high even when the original proxy scale is preserved, confirming thatAνA\_\{\\nu\}is driven by scale and reliability rather than alignment\. Sign\-flipping the proxy makes the actual gain negative and exactAAnegative, whereasAνA\_\{\\nu\}still predicts a large positive signal\.

Table B\.49:Expanded proxy perturbation baselines\. The random\-normal control removes both alignment and the original score distribution; the shuffle control preserves the proxy marginal distribution while destroying itemwise alignment\. In both casesAνA\_\{\\nu\}remains spuriously high\.Figure B\.17:Proxy perturbation stress test\. The bars show realized endpoint gainsΔ⁡\(0\.10\)\\Delta\(0\.10\)under noisy, oracle, shuffled, random\-normal, and sign\-flipped proxy variants\. Coefficient values and baseline prediction errors are reported in Table[B\.49](https://arxiv.org/html/2610.41114#A2.T49); the multi\-dataset extension is Figure[B\.18](https://arxiv.org/html/2610.41114#A2.F18)\. The shuffle condition shows that the result is not a proxy\-scale artifact: preserving the marginal score distribution while destroying itemwise alignment collapses realized gain\.Figure B\.18:Proxy perturbation stress test across eligible analysis slices\. Panel A shows SummEval, Panel B WMT MQM clusters, and Panel C OpenMEVA\-MAGS ROC/WP\. In each panel, the exact first\-order coefficientAAtracks alignment direction across perturbations, while the reliability\-onlyAνA\_\{\\nu\}baseline fails to distinguish aligned from unaligned proxies\.
### B\.19Failure\-attribution guide

A complete report of the diagnostic names the analysis slice, path count, calibration fraction, target construction, proxy source and configuration, planned selection rule, reachable KL radii, tilted variance, RelESS, endpoint and curve\-level errors with their uncertainty, and the baselines used for comparison, includingAA\-only whenever the interval forBBincludes zero\. Large prediction error can indicate coefficient noise, calibration–evaluation shift, finite\-range breakdown, or proxy–gold rank and tail mismatch\. Table[B\.50](https://arxiv.org/html/2610.41114#A2.T50)maps each pattern to its likely source and a recommended follow\-up check \(Section[6](https://arxiv.org/html/2610.41114#S6)\)\.

Table B\.50:Failure\-attribution guide\. The same MAE or sign result can have different causes, so each pattern triggers a different follow\-up check\.
### B\.20Broader impacts

A local forecast of human gain can help identify low\-gain or misaligned proxies before more aggressive optimization, direct human\-evaluation effort to where it changes a decision, and support more conservative deployment choices\. The same diagnostic could also be used to tune stronger optimization against flawed reward models or LLM judges, which could amplify annotator bias, judge\-specific artifacts, or unfair group effects, particularly if a forecast is applied outside its checked window or to a population on which it was not calibrated\. These risks motivate the paper’s emphasis on repeated human calibration, slice\-specific reporting, bounded KL windows, abstention when the evidence is insufficient, and human oversight rather than unconstrained proxy maximization\.

## NeurIPS Paper Checklist

1. 1\.Claims
2. Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
3. Answer:\[Yes\]
4. Justification: The abstract and introduction state the local pre\-optimization diagnostic claim and its local, one\-sided scope, the repeated\-rating calibration setting, the comparison with flexible calibration under matched information, the scope of the GPT\-4o external proxy, the LoRA\-DPO bridge, and the OOD\-transfer result\.
5. Guidelines: - •The answer\[N/A\]means that the abstract and introduction do not include the claims made in the paper\. - •The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations\. A\[No\]or\[N/A\]answer to this question will not be perceived well by the reviewers\. - •The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings\. - •It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper\.
6. 2\.Limitations
7. Question: Does the paper discuss the limitations of the work performed by the authors?
8. Answer:\[Yes\]
9. Justification: Section[6](https://arxiv.org/html/2610.41114#S6)discusses the locality and one\-sidedness of the theory, the checked KL window and how to choose it, the cost of repeated ratings and when to recalibrate, the scope of the judge, reward\-model, Best\-of\-NN, and DPO results, and differences in reproducibility\. Sections[5\.1](https://arxiv.org/html/2610.41114#S5.SS1)–[5\.4](https://arxiv.org/html/2610.41114#S5.SS4)report where flexible calibrators match or beat the coefficient forecast, when the estimate ofBBis unreliable, and where discrete selection and DPO depart from the theorem\.
10. Guidelines: - •The answer\[N/A\]means that the paper has no limitation while the answer\[No\]means that the paper has limitations, but those are not discussed in the paper\. - •The authors are encouraged to create a separate “Limitations” section in their paper\. - •The paper should point out any strong assumptions and how robust the results are to violations of these assumptions \(e\.g\., independence assumptions, noiseless settings, model well\-specification, asymptotic approximations only holding locally\)\. The authors should reflect on how these assumptions might be violated in practice and what the implications would be\. - •The authors should reflect on the scope of the claims made, e\.g\., if the approach was only tested on a few datasets or with a few runs\. In general, empirical results often depend on implicit assumptions, which should be articulated\. - •The authors should reflect on the factors that influence the performance of the approach\. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting\. Or a speech\-to\-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon\. - •The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size\. - •If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness\. - •While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper\. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community\. Reviewers will be specifically instructed to not penalize honesty concerning limitations\.
11. 3\.Theory assumptions and proofs
12. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete \(and correct\) proof?
13. Answer:\[Yes\]
14. Justification: The main text states the local coefficient law and its scope\. Appendix[A](https://arxiv.org/html/2610.41114#A1)gives the assumptions, proofs, finite\-range extensions, and cross\-references for the theoretical results\.
15. Guidelines: - •The answer\[N/A\]means that the paper does not include theoretical results\. - •All the theorems, formulas, and proofs in the paper should be numbered and cross\-referenced\. - •All assumptions should be clearly stated or referenced in the statement of any theorems\. - •The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition\. - •Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material\. - •Theorems and Lemmas that the proof relies upon should be properly referenced\.
16. 4\.Experimental result reproducibility
17. Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper \(regardless of whether the code and data are provided or not\)?
18. Answer:\[Yes\]
19. Justification: Sections[4](https://arxiv.org/html/2610.41114#S4)–[5](https://arxiv.org/html/2610.41114#S5)and Appendix[B](https://arxiv.org/html/2610.41114#A2)describe the tilt construction, proxy–gold paths, calibration/evaluation splits, information regimes, baselines and their tuning, path counts, skipped cases, the DPO setup, and the reported diagnostics\. Appendix[B\.1](https://arxiv.org/html/2610.41114#A2.SS1)distinguishes three levels of reproducibility: released GPT\-4o aggregates reproduce all downstream analyses but not the API calls; scored files and recorded model revisions reproduce the open reward\-model and judge analyses; and the DPO inputs, code, and seeds are released, but the LoRA weights were not retained\. Archived training\-job logs support the reported stage\-level compute accounting\.
20. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •If the paper includes experiments, a\[No\]answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not\. - •If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable\. - •Depending on the contribution, reproducibility can be accomplished in various ways\. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model\. In general\. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model \(e\.g\., in the case of a large language model\), releasing of a model checkpoint, or other means that are appropriate to the research performed\. - •While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution\. For example 1. \(a\)If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm\. 2. \(b\)If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully\. 3. \(c\)If the contribution is a new model \(e\.g\., a large language model\), then there should either be a way to access this model for reproducing the results or a way to reproduce the model \(e\.g\., with an open\-source dataset or instructions for how to construct the dataset\)\. 4. \(d\)We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility\. In the case of closed\-source models, it may be that access to the model is limited in some way \(e\.g\., to registered users\), but it should be possible for other researchers to have some path to reproducing or verifying the results\.
21. 5\.Open access to data and code
22. Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
23. Answer:\[Yes\]
24. Justification: Code, prepared inputs, and derived results are released at[https://github\.com/pacifiic/local\-proxy\-diagnostics](https://github.com/pacifiic/local-proxy-diagnostics), with requirements, run scripts, artifact builders, and the cached GPT\-4o and open\-proxy score files\. This reproduces every reported analysis from prepared inputs\. The exact GPT\-4o API score\-generation run is not reproducible from the aggregate artifact alone, and the DPO LoRA weights were not retained; Appendix[B\.1](https://arxiv.org/html/2610.41114#A2.SS1)describes source\-data access and these limitations\.
25. Guidelines: - •The answer\[N/A\]means that paper does not include experiments requiring code\. - • - •While we encourage the release of code and data, we understand that this might not be possible, so\[No\]is an acceptable answer\. Papers cannot be rejected simply for not including code, unless this is central to the contribution \(e\.g\., for a new open\-source benchmark\)\. - •The instructions should contain the exact command and environment needed to run to reproduce the results\. See the NeurIPS code and data submission guidelines \([https://neurips\.cc/public/guides/CodeSubmissionPolicy](https://neurips.cc/public/guides/CodeSubmissionPolicy)\) for more details\. - •The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc\. - •The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines\. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why\. - •At submission time, to preserve anonymity, the authors should release anonymized versions \(if applicable\)\. - •Providing as much information as possible in supplemental material \(appended to the paper\) is recommended, but including URLs to data and code is permitted\.
26. 6\.Experimental setting/details
27. Question: Does the paper specify all the training and test details \(e\.g\., data splits, hyperparameters, how they were chosen, type of optimizer\) necessary to understand the results?
28. Answer:\[Yes\]
29. Justification: The paper reports datasets, path counts, repeat counts, skipped cases, calibration fractions, fixed target\-heavy transfer weight, KL ranges, baselines and their calibration\-internal tuning grids, DPO hyperparameters and checkpoint selection, and evaluation metrics in the main text and Appendix[B](https://arxiv.org/html/2610.41114#A2)\.
30. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them\. - •The full details can be provided either with the code, in appendix, or as supplemental material\.
31. 7\.Experiment statistical significance
32. Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
33. Answer:\[Yes\]
34. Justification: Table[1](https://arxiv.org/html/2610.41114#S5.T1)a reports path\-level bootstrap 95% intervals, and Figure[2](https://arxiv.org/html/2610.41114#S5.F2)shows path\-cluster bootstrap intervals\. Appendices[B\.7](https://arxiv.org/html/2610.41114#A2.SS7)and[B\.8](https://arxiv.org/html/2610.41114#A2.SS8)give paired path\-clustered bootstrap intervals for every comparison in Tables[1](https://arxiv.org/html/2610.41114#S5.T1)b and[2](https://arxiv.org/html/2610.41114#S5.T2), and the DPO results report the mean and sample SD across three training seeds\. The captions state that GPT\-4o has only four paths, so its intervals are descriptive\.
35. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The authors should answer\[Yes\]if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper\. - •The factors of variability that the error bars are capturing should be clearly stated \(for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions\)\. - •The method for calculating the error bars should be explained \(closed form formula, call to a library function, bootstrap, etc\.\) - •The assumptions made should be given \(e\.g\., Normally distributed errors\)\. - •It should be clear whether the error bar is the standard deviation or the standard error of the mean\. - •It is OK to report 1\-sigma error bars, but one should state it\. The authors should preferably report a 2\-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified\. - •For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range \(e\.g\., negative error rates\)\. - •If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text\.
36. 8\.Experiments compute resources
37. Question: For each experiment, does the paper provide sufficient information on the computer resources \(type of compute workers, memory, time of execution\) needed to reproduce the experiments?
38. Answer:\[Yes\]
39. Justification: Appendix[B\.1](https://arxiv.org/html/2610.41114#A2.SS1)reports the CPU and memory configuration, eight NVIDIA B200 GPUs, one GPU per worker, CPU reproduction costs, and logged GPU runtimes for candidate generation, DPO training, and open\-judge scoring\. The accounting is stage\-level rather than a complete per\-experiment profile of end\-to\-end runtime and peak memory\.
40. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage\. - •The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute\. - •The paper should disclose whether the full research project required more compute than the experiments reported in the paper \(e\.g\., preliminary or failed experiments that didn’t make it into the paper\)\.
41. 9\.Code of ethics
43. Answer:\[Yes\]
44. Justification: The paper analyzes existing evaluation datasets and derived diagnostics\. It does not introduce a deployed system\. It frames external\-judge results as a scoped external\-proxy evaluation rather than a deployment recommendation\.
45. Guidelines: - •The answer\[N/A\]means that the authors have not reviewed the NeurIPS Code of Ethics\. - •If the authors answer\[No\], they should explain the special circumstances that require a deviation from the Code of Ethics\. - •The authors should make sure to preserve anonymity \(e\.g\., if there is a special consideration due to laws or regulations in their jurisdiction\)\.
46. 10\.Broader impacts
47. Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
48. Answer:\[Yes\]
49. Justification: Appendix[B\.20](https://arxiv.org/html/2610.41114#A2.SS20)discusses both positive and negative broader impacts of the work\. On the positive side, a local proxy\-optimization diagnostic can help identify low\-gain or misaligned proxies before more aggressive optimization, target human\-evaluation effort more efficiently, and support more conservative deployment decisions\. On the negative side, the same diagnostic could be used to tune stronger optimization against flawed reward models or LLM judges, which could amplify annotator bias, judge\-specific artifacts, or unfair group effects\. These risks motivate the paper’s emphasis on repeated human calibration, slice\-specific reporting, bounded KL windows, and human oversight rather than unconstrained proxy maximization\.
50. Guidelines: - •The answer\[N/A\]means that there is no societal impact of the work performed\. - •If the authors answer\[N/A\]or\[No\], they should explain why their work has no societal impact or why the paper does not address societal impact\. - •Examples of negative societal impacts include potential malicious or unintended uses \(e\.g\., disinformation, generating fake profiles, surveillance\), fairness considerations \(e\.g\., deployment of technologies that could make decisions that unfairly impact specific groups\), privacy considerations, and security considerations\. - •The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments\. However, if there is a direct path to any negative applications, the authors should point it out\. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation\. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster\. - •The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from \(intentional or unintentional\) misuse of the technology\. - •If there are negative societal impacts, the authors could also discuss possible mitigation strategies \(e\.g\., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML\)\.
51. 11\.Safeguards
52. Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse \(e\.g\., pre\-trained language models, image generators, or scraped datasets\)?
53. Answer:\[N/A\]
54. Justification: The paper does not release a high\-risk model, scraped dataset, or deployed system requiring controlled access safeguards\.
55. Guidelines: - •The answer\[N/A\]means that the paper poses no such risks\. - •Released models that have a high risk for misuse or dual\-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters\. - •Datasets that have been scraped from the Internet could pose safety risks\. The authors should describe how they avoided releasing unsafe images\. - •We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort\.
56. 12\.Licenses for existing assets
57. Question: Are the creators or original owners of assets \(e\.g\., code, data, models\), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
58. Answer:\[Yes\]
59. Justification: Appendix[B\.1](https://arxiv.org/html/2610.41114#A2.SS1), Table[B\.1](https://arxiv.org/html/2610.41114#A2.T1), lists the source, license or access terms, and redistribution status for each external asset used in the paper\. The table includes SummEval \(MIT\), WMT MQM \(Apache\-2\.0\), RewardBench \(repository Apache\-2\.0; dataset card ODC\-BY\), MT\-Bench/FastChat \(repository Apache\-2\.0;mt\_bench\_human\_judgmentsdataset card CC\-BY\-4\.0\), UltraFeedback \(MIT\), and the model weights used for scoring and training: Skywork\-Reward\-Llama\-3\.1\-8B\-v0\.2 \(Skywork Community License\), ArmoRM\-Llama3\-8B\-v0\.1 \(Llama 3 license\), OpenAssistant reward\-model\-deberta\-v3\-large\-v2 \(MIT\), Qwen2\.5\-32B\-Instruct \(Apache\-2\.0\), and Qwen2\.5\-3B\-Instruct \(Qwen Research License\)\. Model weights are not redistributed\. For assets without a verified public license page for the source used here, such as OpenMEVA\-MAGS ROC/WP and the QGEval human files, the paper marks the license/terms as not specified and does not redistribute the raw data\.
60. Guidelines: - •The answer\[N/A\]means that the paper does not use existing assets\. - •The authors should cite the original paper that produced the code package or dataset\. - •The authors should state which version of the asset is used and, if possible, include a URL\. - •The name of the license \(e\.g\., CC\-BY 4\.0\) should be included for each asset\. - •For scraped data from a particular source \(e\.g\., website\), the copyright and terms of service of that source should be provided\. - •If assets are released, the license, copyright information, and terms of use in the package should be provided\. For popular datasets,[paperswithcode\.com/datasets](https://paperswithcode.com/datasets)has curated licenses for some datasets\. Their licensing guide can help determine the license of a dataset\. - •For existing datasets that are re\-packaged, both the original license and the license of the derived asset \(if it has changed\) should be provided\. - •If this information is not available online, the authors are encouraged to reach out to the asset’s creators\.
61. 13\.New assets
62. Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
63. Answer:\[Yes\]
64. Justification: The public release documents the analysis scripts, generated diagnostics, artifact builders, requirements, run scripts, checksums, and cached proxy scores, under an MIT license for the code\. The cached scores support reproduction of the reported analyses, while exact reruns of GPT\-4o score generation would require raw per\-call logs beyond the released aggregate file\. Appendix[B\.1](https://arxiv.org/html/2610.41114#A2.SS1)summarizes these assets\.
65. Guidelines: - •The answer\[N/A\]means that the paper does not release new assets\. - •Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates\. This includes details about training, license, limitations, etc\. - •The paper should discuss whether and how consent was obtained from people whose asset is used\. - •At submission time, remember to anonymize your assets \(if applicable\)\. You can either create an anonymized URL or include an anonymized zip file\.
66. 14\.Crowdsourcing and research with human subjects
67. Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation \(if any\)?
68. Answer:\[N/A\]
69. Justification: The paper analyzes pre\-existing human annotation datasets and does not conduct new crowdsourcing or new human\-subject data collection\.
70. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper\. - •According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector\.
71. 15\.Institutional review board \(IRB\) approvals or equivalent for research with human subjects
72. Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board \(IRB\) approvals \(or an equivalent approval/review based on the requirements of your country or institution\) were obtained?
73. Answer:\[N/A\]
74. Justification: The paper uses secondary analysis of existing annotations and does not introduce new human\-subject interaction\.
75. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Depending on the country in which research is conducted, IRB approval \(or equivalent\) may be required for any human subjects research\. If you obtained IRB approval, you should clearly state this in the paper\. - •We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution\. - •For initial submissions, do not include any information that would break anonymity \(if applicable\), such as the institution conducting the review\.
76. 16\.Declaration of LLM usage
77. Question: Does the paper describe the usage of LLMs if it is an important, original, or non\-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does*not*impact the core methodology, scientific rigor, or originality of the research, declaration is not required\.
78. Answer:\[Yes\]
79. Justification: LLMs are used as proxies under evaluation, not as part of the method: a GPT\-4o judge, a Qwen2\.5\-32B\-Instruct judge, LLM\-based reward models, and a Qwen2\.5\-3B\-Instruct policy for the DPO bridge\. Section[5](https://arxiv.org/html/2610.41114#S5)and Appendices[B\.14](https://arxiv.org/html/2610.41114#A2.SS14),[B\.9](https://arxiv.org/html/2610.41114#A2.SS9),[B\.10](https://arxiv.org/html/2610.41114#A2.SS10), and[B\.16](https://arxiv.org/html/2610.41114#A2.SS16)describe the models, revisions, prompts or scoring settings, and the fields that are not recoverable from stored artifacts\.
80. Guidelines: - •The answer\[N/A\]means that the core method development in this research does not involve LLMs as any important, original, or non\-standard components\. - •Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described\.