Better Policies, Worse Verifier Audits

# Better Policies, Worse Verifier Audits
Source: [https://arxiv.org/html/2610.29029](https://arxiv.org/html/2610.29029)
Daniel Cieślak††thanks:Corresponding author:daniel\.cieslak@pg\.edu\.plJacek ŁubińskiAffiliation:Gdańsk University of TechnologyAffiliation:Gdańsk, Poland

###### Abstract

Verifier stacks increasingly sit inside post\-training and test\-time compute loops\. We study a specific control loop: a component verifier is evaluated on fresh policy outputs against the*full*success label and refreshed when that aggregate score falls\. For a conjunctive targety=F∧My=F\\wedge M, its on\-policy AUC is\(1−q\)​Aown\+q​Across\(1\-q\)A\_\{\\rm own\}\+qA\_\{\\rm cross\}, whereq=Pr⁡\[F=1∣y=0\]q=\\Pr\[F=1\\mid y=0\]\. As the policy mastersFF,qqcan rise toward one, removing weight fromFF\-failures and rewarding information aboutMM\. We call this*on\-policy audit collapse*\. We derive a sharp identified set, a complete\-erasure threshold, and the exact point at which the audit can prefer a worse component verifier; its population optimum is the full\-outcome predictor, not the component oracle\. An exact oracle exhibits the collapse, a natural multi\-label text control shows inversion can select a head with lower canonicalFF\-AUC at the observed failure mixture, and a prediction\-registered ECG study shows that acting on the audit can reduce off\-policyFF\-AUC, transfer the damage to an unseen generator, and increase downstream constraint violations\. Preserving own\-failure support prevents the damage\. Matching label\-complexity bounds and a controlled\-mixture audit give the corresponding repair: audit the component on its own failures, and keep those failures in the refresh distribution\.

## 1Introduction

Verifier stacks increasingly sit inside the optimization loop\. Post\-training systems combine outcome, process, rubric and constraint signals, while test\-time compute uses verifier scores to search, rank and select candidates\([Cobbe et al\., 2021](https://arxiv.org/html/2610.29029#bib.bib1);[Lightman et al\., 2024](https://arxiv.org/html/2610.29029#bib.bib2);[Brown et al\., 2024](https://arxiv.org/html/2610.29029#bib.bib3);[Snell et al\., 2025](https://arxiv.org/html/2610.29029#bib.bib4);[Mahmoud et al\., 2026](https://arxiv.org/html/2610.29029#bib.bib5);[Wu et al\., 2023](https://arxiv.org/html/2610.29029#bib.bib6);[Zhou et al\., 2024](https://arxiv.org/html/2610.29029#bib.bib7);[Moskovitz et al\., 2024](https://arxiv.org/html/2610.29029#bib.bib8);[Liu et al\., 2026](https://arxiv.org/html/2610.29029#bib.bib9);[Bai et al\., 2026](https://arxiv.org/html/2610.29029#bib.bib10)\)\. A natural but unsafe control loop is to monitor a component verifier on fresh policy samples against the full task outcome; if that aggregate score drops, collect new on\-policy data and refresh the verifier\. Policy\-interaction data are in fact used to refine reward models and iterative RLHF pipelines repeatedly update preference models on fresh policy data\([Bai et al\., 2022](https://arxiv.org/html/2610.29029#bib.bib11);[Sun et al\., 2025](https://arxiv.org/html/2610.29029#bib.bib12)\)\. This paper shows that this loop can become systematically wrong*because the policy improved*\.

Let success requirey=F∧My=F\\wedge Mand letsFs\_\{F\}be a verifier intended to scoreFF\. Among failures, some violateFFand others already satisfyFFbut fail onlyMM\. Pairwise additivity of AUC gives

AUC⁡\(sF;y\)=\(1−q\)​Aown\+q​Across,q=Pr⁡\[F=1∣y=0\]\.\\operatorname\{AUC\}\(s\_\{F\};y\)=\(1\-q\)A\_\{\\rm own\}\+qA\_\{\\rm cross\},\\qquad q=\\Pr\[F=1\\mid y=0\]\.\(1\)HereAownA\_\{\\rm own\}is the audit’s*own\-failure separability*: target successes versusFF\-failures\. It is not, in general, identical to canonicalFF\-AUC; Section[2](https://arxiv.org/html/2610.29029#S2)separates the two estimands explicitly\.AcrossA\_\{\\rm cross\}compares successes with failures that already satisfyFF; an exactFF\-oracle hasAcross=1/2A\_\{\\rm cross\}=1/2by construction\. When optimization increases satisfaction ofFFfaster than the remaining conjunct—in particular asFFsaturates whileMMremains imperfect—qqrises\. Consequently, the standard aggregate monitor progressively removes weight from the verifier’s own failure mode and places it on information about the*other*conjunct\. The paradox is therefore directional:the more completely the policy masters a component, the less the usual on\-policy audit measures that component\.We call this*on\-policy audit collapse*: optimization removes the very failure mode needed to evaluate the module that learned it\.

This creates a concrete failure chain\. First, the monitor loses identification: a falling AUC need not imply a worse component verifier\. Second, it can become actively anti\-aligned with the audit’s intended subproblem: at a calculableq∗q^\{\\ast\}it prefers a head with worseAownA\_\{\\rm own\}because that head leaks information aboutMM; in our text and engine controls this can also select a head with worse canonicalFF\-AUC\. Third, the usual response to a bad monitor—on\-policy refresh—can remove component competence\. Once optimization has depletedFF\-failures, refresh data alone no longer identify the boundary the component verifier must retain; outcome labels can actively entangle it, while even correctFFlabels cannot reconstruct failures absent from support\. The result can look benign on the monitored distribution while becoming worse exactly on the off\-support failures a constraint checker must catch after a generator, prompt distribution, or policy shift\.

policy learnsFFFF\-failures vanishq↑q\\\!\\uparrowaudit changes meaningcross\-signal winsrefresh canforgetFF

Viewed as a learning system, this is a control\-plane failure: optimizing the policy changes the data used to decide whether its verifier should be replaced or refreshed\.

#### When does the failure apply?

The central result requires three ingredients: a head intended for componentFF, an*on\-policy*audit against the full outcomeyyrather than a component\-specific label, and use of that audit to compare, early\-stop or trigger refreshes\. If trustedFFlabels are available and the component is audited directly \(preferably on a reference distribution that preservesFF\-failures\), the collapse in Eq\.[1](https://arxiv.org/html/2610.29029#S1.E1)is avoided\. Our claim is therefore about this specific but natural interface, not about every verifier evaluation protocol\.

#### Why this interface is current\.

Its ingredients already coexist in modern post\-training systems\. Rubric RL independently scores prompt\-specific criteria and aggregates them into a scalar on\-policy reward\([Gunjal et al\., 2026](https://arxiv.org/html/2610.29029#bib.bib42);[Mahmoud et al\., 2026](https://arxiv.org/html/2610.29029#bib.bib5)\); policy\-aware rubric work reports that substantial criterion mass is already saturated or dead under current policies\([Tyagi et al\., 2026](https://arxiv.org/html/2610.29029#bib.bib43)\); and fresh policy\-interaction data are used to refine reward models or preference models\([Sun et al\., 2025](https://arxiv.org/html/2610.29029#bib.bib12);[Bai et al\., 2022](https://arxiv.org/html/2610.29029#bib.bib11)\)\. We do*not*claim that these systems implement our exact audit\-trigger\-refresh controller end to end\. They establish that criterion\-level verifiers, policy\-induced saturation, scalar aggregation and on\-policy refinement are contemporary ingredients; Appendix[AD\.3](https://arxiv.org/html/2610.29029#A30.SS3)maps the interfaces explicitly\.

Our experiments are organized to test this chain rather than to accumulate unrelated benchmarks\. The engine domain gives an exact component oracle, allowing us to separate real verifier failure from a case\-mix artifact\. GSM8K supplies a familiar language\-domain negative control for ranking inversion\. A natural multi\-label GoEmotions stress test uses separately annotated semantic criteria to test audit collapse, inversion and refresh under support depletion; a constructed AG News control is reported in the appendix\. ECG then stresses the operational consequence on a frozen external benchmark with three generator families, subject\-held\-out splits, two refresh model classes, downstream best\-of\-1616selection, and a precommitted set of predictions\. Across these settings the observations line up with the mechanism: exact\-oracle AUC collapses, the analytic inversion threshold predicts where inversion does and does not occur, optimization removes own\-failure support, outcome refresh damages off\-policy canonicalFF\-AUC and downstream constraints in every paired split, and deliberately restoring failure support prevents the damage\.

#### Contributions\.

\(1\) On\-policy audit collapse\.We derive the mixture law, a sharp identified set, and the complete\-erasure thresholdq≥max⁡\(A,1−A\)q\\geq\\max\(A,1\-A\)\(Section[4](https://arxiv.org/html/2610.29029#S4)\)\.\(2\) Audit inversion\.We give the thresholdq∗q^\{\\ast\}at which cross\-component signal overtakes own\-failure separability and show that the aggregate audit’s population maximizer is the full\-outcome predictor, not the component oracle \(Proposition[2](https://arxiv.org/html/2610.29029#Thmproposition2)\)\.\(3\) The refresh trap\.A controlled semantic\-language stress test and a prediction\-registered ECG study separate label mismatch from support mismatch and connect refresh to downstream harm \(Section[5](https://arxiv.org/html/2610.29029#S5)\)\.\(4\) Support\-preserving repair\.We give matching passive label\-complexity bounds and a designed two\-mixture audit whose error does not depend on the deployed policy’s saturation \(Section[6](https://arxiv.org/html/2610.29029#S6)\)\. Downstream selection/training\-interface results are deliberately kept as a boundary analysis in the appendix\.

## 2Mechanism\-first evaluation

We test the chain implied by Eq\.[1](https://arxiv.org/html/2610.29029#S1.E1), not a collection of interchangeable benchmarks\. Engine supplies an exact oracle and matched pools for the inversion threshold; GSM8K is a language\-domain negative control where inversion should not occur; GoEmotions supplies two separately annotated semantic criteria; and ECG is the frozen operational stress test with three generator families, subject\-held\-out evaluation, two refresh model classes and a downstream gate\. Protocols and domain\-specific details are in the appendices\.

In every domainy=F∧My=F\\wedge M\. We keep two component estimands distinct\. The theory isolates*own\-failure separability*Aown=AUC\(sF;F=1,M=1vsF=0\)A\_\{\\rm own\}=\\operatorname\{AUC\}\(s\_\{F\};F\{=\}1,M\{=\}1\\text\{ vs \}F\{=\}0\), because these are exactly the outcome\-audit pairs whose negative is anFF\-failure\. Canonical component quality is insteadAF=AUC\(sF;F=1vsF=0\)A\_\{F\}=\\operatorname\{AUC\}\(s\_\{F\};F\{=\}1\\text\{ vs \}F\{=\}0\)\. They need not agree: ifρ=Pr⁡\[M=1∣F=1\]\\rho=\\Pr\[M=1\\mid F=1\]andAaux=AUC\(sF;F=1,M=0vsF=0\)A\_\{\\rm aux\}=\\operatorname\{AUC\}\(s\_\{F\};F\{=\}1,M\{=\}0\\text\{ vs \}F\{=\}0\), then

AF=ρ​Aown\+\(1−ρ\)​Aaux\.A\_\{F\}=\\rho A\_\{\\rm own\}\+\(1\-\\rho\)A\_\{\\rm aux\}\.\(2\)Equation[2](https://arxiv.org/html/2610.29029#S2.E2)is just the law of total probability overMMwithin the canonical positive classF=1F=1; no independence assumption is used\. Propositions[1](https://arxiv.org/html/2610.29029#Thmproposition1)–[2](https://arxiv.org/html/2610.29029#Thmproposition2)characterize what the*outcome*audit identifies and rewards aboutAownA\_\{\\rm own\}; whenever we claim actual component degradation we reportAFA\_\{F\}separately\. Best\-of\-KKsupply, selection efficiency and Answer@KKare defined and audited in Appendix[AC](https://arxiv.org/html/2610.29029#A29)\.

## 3The paradox: policy improves, audit collapses

Table 1:The policy improves under the oracle target while the component head’s full\-target AUC falls toward chance\. The component is feasibility for engine and boxed format for GSM8K\. Engine:K=16K=16, three seeds,1 6001\\,600pairs \(the original round; the matched study of Appendix[AF](https://arxiv.org/html/2610.29029#A32)uses4 0004\\,000pairs\)\. GSM8K:500500prompts,K=16K=16prefix of6464, three seeds\.The first empirical fact is intentionally paradoxical\. In engine, oracle target success per draw rises3\.6×3\.6\\timeswhile the feasibility head’s aggregate AUC falls from0\.800\.80to0\.510\.51\. In GSM8K, target success rises2\.1×2\.1\\timeswhile the format head falls from0\.920\.92to0\.510\.51\. At the same time the native selectors fail to convert the newly available supply into selected success: selection efficiency falls to0\.050\.05and0\.090\.09\. If aggregate on\-policy AUC were a direct measure of component quality, these numbers would suggest that both components deteriorated sharply just as the policies learned to satisfy them\.

#### The exact\-oracle control rules out that interpretation\.

The component itself remains available\. Two unused engine critics with different seeds show the same on\-policy collapse \(0\.450\.45–0\.580\.58\) while their mass heads stay at0\.960\.96–0\.980\.98; the feasibility head retains own\-property AUC0\.9400\.940–0\.9910\.991on matched DPO pools, and the GSM8K format head retains0\.9950\.995accuracy against unboxed failures\. More strongly, the next section shows that an*exact*component oracle exhibits the same aggregate symptom\. Thus the collapse cannot, by itself, identify component degradation\. It is the empirical signature predicted by a changing failure mixture\.

## 4On\-policy audit collapse

For one candidate poolrr, every negative example belongs to one of two semantic failure bins: it either fails the scored conditionFF, or it satisfiesFFand failsMM\. Pairwise additivity of AUC therefore gives

Ar=\(1−qr\)Aown,r\+qrAcross,r,qr=Pr\[F=1∣y=0,r\]\.A\_\{r\}=\(1\-q\_\{r\}\)A\_\{\{\\rm own\},r\}\+q\_\{r\}A\_\{\{\\rm cross\},r\},\\qquad q\_\{r\}=\\Pr\[F=1\\mid y=0,r\]\.\(3\)HereAown,rA\_\{\{\\rm own\},r\}is the audit’s own\-failure separability: target successes versus failures ofFF\. Equation[2](https://arxiv.org/html/2610.29029#S2.E2)distinguishes this audit estimand from canonicalFF\-AUC\. The second term,Across,rA\_\{\{\\rm cross\},r\}, asks the same head to order target successes against failures that already satisfyFF\. Asqrq\_\{r\}grows, the aggregate audit places less weight on the head’s own subproblem\.

All statements below are per pool; becauseA¯≠\(1−q¯\)​A¯own\+q¯​A¯cross\\bar\{A\}\\neq\(1\-\\bar\{q\}\)\\bar\{A\}\_\{\\rm own\}\+\\bar\{q\}\\bar\{A\}\_\{\\rm cross\}in general, every reconstruction averages the per\-pool identity \(Section[2](https://arxiv.org/html/2610.29029#S2)\)\.

###### Proposition 1\(Identified set for own\-failure separability\)\.

Partition negatives into semantic failure binsB1,…,BJB\_\{1\},\\ldots,B\_\{J\}, withπj=Pr⁡\[Bj∣y=0\]\\pi\_\{j\}=\\Pr\[B\_\{j\}\\mid y=0\]andaj=AUC⁡\(s,y=1​vs​Bj\)a\_\{j\}=\\operatorname\{AUC\}\(s;y=1\\text\{ vs \}B\_\{j\}\)\. Given only aggregate AUCAAand the observed bin weightsπ\\pi, the sharp identified set is

ℐ⁡\(A,π\)=\{a∈\[0,1\]J:π⊤​a=A\}\.\\mathcal\{I\}\(A,\\pi\)=\\\{a\\in\[0,1\]^\{J\}:\\pi^\{\\top\}a=A\\\}\.For the two\-bin split in Eq\.[3](https://arxiv.org/html/2610.29029#S4.E3), ifq<1q<1,

Aown∈\[max⁡\(0,A−q1−q\),min⁡\(1,A1−q\)\]\.A\_\{\\rm own\}\\in\\left\[\\max\\\!\\left\(0,\\frac\{A\-q\}\{1\-q\}\\right\),\\min\\\!\\left\(1,\\frac\{A\}\{1\-q\}\\right\)\\right\]\.\(4\)This interval is all of\[0,1\]\[0,1\]exactly when

q≥max⁡\(A,1−A\)=12\+\|A−12\|\.q\\geq\\max\(A,1\-A\)=\\tfrac\{1\}\{2\}\+\\left\|A\-\\tfrac\{1\}\{2\}\\right\|\.\(5\)The bounds are attainable\. If the bin\-conditional score distributions are held fixed and aggregate AUC is measured under known mixture rows ofΠ\\Pi, then the vectoraais point\-identified for every compatible value iffrank⁡\(Π\)=J\\operatorname\{rank\}\(\\Pi\)=J; with two bins, two distinct mixture weights suffice\. If the scores, but not the bins, of all negatives are also observed, the sharp interval forAownA\_\{\\rm own\}becomes a trimming bound\([Horowitz and Manski, 1995](https://arxiv.org/html/2610.29029#bib.bib15)\): the means of the lowest and of the highest\(1−q\)\(1\-q\)\-fraction of the valuesh⁡\(x\)h\(x\), the share of positives ranked above negativexx\(Appendix[AB](https://arxiv.org/html/2610.29029#A28)\)\.

#### More than two conjuncts\.

ForJ\>2J\>2semantic failure modes the same object is simplyA=∑jπj​ajA=\\sum\_\{j\}\\pi\_\{j\}a\_\{j\}\. Mastering one component shrinks the weight of its own\-failure bin and reallocates audit mass to whichever other failure modes remain\. There is no privileged scalarqqin the multi\-bin case, but the identification question is unchanged: the auditor needs either direct bin labels or enough deliberately distinct mixture rows for the resulting mixture matrix to have full column rank\.

#### What complete erasure means\.

Equation[5](https://arxiv.org/html/2610.29029#S4.E5)is stronger than saying that AUC is sensitive to case mix\. Even after the auditor observes the mixture weightqqexactly, the aggregate statistic can contain*zero*information aboutAownA\_\{\\rm own\}\. Every matched engine row in Table[2](https://arxiv.org/html/2610.29029#S4.T2)is in this regime:q≥0\.916q\\geq 0\.916and aggregate AUC0\.490\.49–0\.580\.58are compatible with everyAown∈\[0,1\]A\_\{\\rm own\}\\in\[0,1\], while direct bin audits measure0\.940\.94–0\.990\.99\.

#### What does a realistic auditor know?

An auditor usually has outcome labels and scores for every candidate but failure causes only on a labelled subsample, from whichqqis estimated\. This extra information barely helps: on the ECG base test pools, observing every negative’s score lowers the share of erased pools only from37%37\\%to34%34\\%\(Llama\) and from48%48\\%to48%48\\%\(Phi\), and estimatingqqfrom5050labels leaves36%36\\%and48%48\\%, because in small pools mosthh\-values sit at00or11\(Appendix[AB](https://arxiv.org/html/2610.29029#A28)\)\.

The same point holds without a learned verifier\. The exact oraclesF=Fs\_\{F\}=FhasAUC⁡\(F,y\)=1−q/2\\operatorname\{AUC\}\(F;y\)=1\-q/2: on stored pools this perfect head falls from0\.8330\.833to0\.5150\.515\(engine\) and from0\.9220\.922to0\.5190\.519\(GSM8K\), realizing a composition\-only path \(Aown=1A\_\{\\rm own\}=1,Across=1/2A\_\{\\rm cross\}=1/2\)\. Section[7](https://arxiv.org/html/2610.29029#S7)gives the opposite control, genuine critic exploitation, with the same aggregate symptom\.

Table 2:Chance\-level aggregate AUC can coexist with near\-perfect ordering of the component’s own failures\.qqis the share of off\-target candidates that already satisfy the scored conjunct\. ECG rows are frozen pre\-DPO test pools; Gemma’s has no informative target\-positive requirement and is omitted rather than imputed\.Figure 1:Audit collapse\.Left: aggregate outcome AUC changes systematically with the failure mixtureqqeven when the component’s own\-bin ordering remains strong\. Right: the zero\-parameterAcross=1/2A\_\{\\rm cross\}=1/2reconstruction tracks measured AUC\. The complete\-erasure region of Eq\.[5](https://arxiv.org/html/2610.29029#S4.E5)is shown separately in Appendix[P](https://arxiv.org/html/2610.29029#A16)\.
#### External check in a non\-extreme regime\.

On ECG the deployability head is effectively exact \(Aown=1\.000A\_\{\\rm own\}=1\.000\), yet the aggregate reads0\.7510\.751and0\.7910\.791atq=0\.61q=0\.61–0\.670\.67\(Table[2](https://arxiv.org/html/2610.29029#S4.T2)\); the mean per\-pool interval from\(Ar,qr\)\(A\_\{r\},q\_\{r\}\)is\[0\.47,1\]\[0\.47,1\]and\[0\.45,1\]\[0\.45,1\], with37%37\\%and48%48\\%of pools erased\.

### 4\.1Audit inversion: cross\-component signal can win

Non\-identification says the aggregate cannot certify a component\. Equation[3](https://arxiv.org/html/2610.29029#S4.E3)says more: it states what the aggregate*rewards*when used to compare, select or refresh verifiers\.

###### Proposition 2\(Audit inversion and its maximizer\)\.

Fix a pool with mixtureqq\.\(i\)For two headsf,gf,g,A⁡\(g\)−A⁡\(f\)=\(1−q\)​\(agown−afown\)\+q⁡\(agcross−afcross\)A\(g\)\-A\(f\)=\(1\-q\)\(a^\{\\rm own\}\_\{g\}\-a^\{\\rm own\}\_\{f\}\)\+q\(a^\{\\rm cross\}\_\{g\}\-a^\{\\rm cross\}\_\{f\}\)\. Ifffhas better own\-failure separability \(Δown=afown−agown\>0\\Delta\_\{\\rm own\}=a^\{\\rm own\}\_\{f\}\-a^\{\\rm own\}\_\{g\}\>0\) butggseparatesFF\-satisfying candidates byMMbetter \(Δcross=agcross−afcross\>0\\Delta\_\{\\rm cross\}=a^\{\\rm cross\}\_\{g\}\-a^\{\\rm cross\}\_\{f\}\>0\), the audit prefersggiff

q\>q∗=ΔownΔown\+Δcross,q\>q^\{\\ast\}=\\frac\{\\Delta\_\{\\rm own\}\}\{\\Delta\_\{\\rm own\}\+\\Delta\_\{\\rm cross\}\},and forΔown=0\\Delta\_\{\\rm own\}=0it prefersggat everyq\>0q\>0\.\(ii\)Over all scoress⁡\(x\)s\(x\)of the candidate, the pool’s aggregate AUC is maximized by any strictly increasing transform ofηy​\(x\)=Pr⁡\[y=1∣x\]\\eta\_\{y\}\(x\)=\\Pr\[y=1\\mid x\]\.\(iii\)IfF=ϕ⁡\(x\)F=\\phi\(x\)is computable from the candidate, thenaown​\(ηy\)=1a^\{\\rm own\}\(\\eta\_\{y\}\)=1, and the exact oracles=ϕs=\\phisatisfiesA⁡\(ηy\)−A⁡\(ϕ\)=q⁡\(across​\(ηy\)−12\)≥0A\(\\eta\_\{y\}\)\-A\(\\phi\)=q\\,\\bigl\(a^\{\\rm cross\}\(\\eta\_\{y\}\)\-\\tfrac\{1\}\{2\}\\bigr\)\\geq 0: the outcome predictor weakly dominates the perfect component verifier at everyqq, strictly unlessηy\\eta\_\{y\}is constant onFF\-satisfying candidates\.

Part \(i\) is Eq\.[3](https://arxiv.org/html/2610.29029#S4.E3)differenced; \(ii\) is the Neyman–Pearson property of AUC\([Clémençon et al\., 2008](https://arxiv.org/html/2610.29029#bib.bib13)\); \(iii\) follows becausey=1y=1impliesF=1F=1, soηy=0\\eta\_\{y\}=0on everyFF\-failure while the oracle is constant on\{F=1\}\\\{F=1\\\}\(Appendix[AD](https://arxiv.org/html/2610.29029#A30)\)\. The content is not the arithmetic but its use\. If aggregate on\-policy AUC is used to*compare or early\-stop*component heads, its selection pressure favors scores increasingly aligned withηy\\eta\_\{y\}, hence with information about the rest of the target\. A subsequent outcome\-labelled refresh is a distinct training intervention, studied in Section[5](https://arxiv.org/html/2610.29029#S5); it need not literally optimize AUC to inherit the same entangling signal\. The within/cross split mirrors the xAUC decomposition used for group fairness\([Kallus and Zhou, 2019](https://arxiv.org/html/2610.29029#bib.bib16)\); here the groups are semantic failure bins and the question is which head the audit statistic prefers\.

#### The threshold predicts where inversion appears\.

On matched 1\.5B sum\-DPO engine pools the learned feasibility head has\(aown,across\)=\(0\.966,0\.519\)\(a^\{\\rm own\},a^\{\\rm cross\}\)=\(0\.966,0\.519\)and the exact oracle\(1,12\)\(1,\\tfrac\{1\}\{2\}\), soq∗=0\.64q^\{\\ast\}=0\.64; at the observedq=0\.970q=0\.970the audit ranks the oracle*below*the imperfect head \(0\.5150\.515vs0\.5320\.532\)\. The oracle also has canonicalFF\-AUC11by definition, while direct feasibility AUC of the learned heads is0\.9400\.940–0\.9910\.991on the matched DPO pools \(Appendix[AC](https://arxiv.org/html/2610.29029#A29)\), so this is not merely a relabelling ofAownA\_\{\\rm own\}\. On GSM8K the format head’sacross=0\.489<12a^\{\\rm cross\}=0\.489<\\tfrac\{1\}\{2\}, so noq∗q^\{\\ast\}exists, and indeed the oracle stays above it \(0\.5180\.518vs0\.5060\.506; Appendix[C](https://arxiv.org/html/2610.29029#A3)\)\. A holistic on\-target engine classifier, the extreme of entanglement, avoids the component\-AUC cliff yet captures onlyη=0\.49\\eta=0\.49–0\.600\.60of the best\-of\-KKlift versus0\.900\.90–0\.940\.94for the gated two\-head critic \(Appendix[AC](https://arxiv.org/html/2610.29029#A29)\): the head favored by the aggregate audit is not the best component representation for the selector\.

#### Optimization drives audits into this regime\.

On the post\-DPO ECG test pools, scored by the same frozen critic, the within\-pool own\-failure bin of the deployability head is empty in every evaluable pool of the four evaluable Llama gate/inference runs and in6464–72%72\\%of evaluable pools of the six Phi gate/inference runs, up from13%13\\%and36%36\\%at base\. There no passive audit recoversAownA\_\{\\rm own\}from the deployed mixture \(Section[6](https://arxiv.org/html/2610.29029#S6)\), and the on\-policy refresh data no longer identify how the head should rank the missing own\-failure mode\. The next section tests what happens when a refresh procedure acts on that depleted support\.

## 5The refresh trap: acting on audit collapse

Proposition[2](https://arxiv.org/html/2610.29029#Thmproposition2)describes what the aggregate audit prefers\. Deployment makes the problem operational: a low monitor value can trigger collection of fresh policy samples and a verifier refresh\. The mixture law predicts two distinct hazards\.Label mismatch:if a component verifier is refreshed against the full outcomeyy, optimization explicitly moves it toward the audit’s preferred entangled predictor\.Support mismatch:even with the correct component labelsFF, a saturated policy may provide almost noFF\-failures, so the refresh data contain little or no information about the decision boundary the verifier must retain\. The second hazard means the problem is not fixed simply by choosing the “right” label\.

#### Natural multi\-label semantic stress test\.

On GoEmotions\([Demszky et al\., 2020](https://arxiv.org/html/2610.29029#bib.bib41)\), keepFFas absence of*annoyance*and use a post\-hoc, transparently reported high\-qqpositive control withMM= absence of*gratitude*\. At the naturally observedq=\.522q=\.522, the component specialist has canonicalFF\-AUC\.679\.679but an aggregate audit of only\.442\.442\. The deliberately entangled scorez⁡\(sF\)\+z⁡\(sM\)z\(s\_\{F\}\)\+z\(s\_\{M\}\)has substantially*worse*canonicalFF\-AUC \(\.570\.570\) yet the aggregate audit prefers it \(\.681\.681; bootstrap 95% difference\[\.210,\.270\]\[\.210,\.270\]\), with predicted crossingq∗=\.126q^\{\*\}=\.126\. Sweeping leakage strengthsλ=z⁡\(sF\)\+λ​z​\(sM\)s\_\{\\lambda\}=z\(s\_\{F\}\)\+\\lambda z\(s\_\{M\}\)produces the same quantitative tradeoff: as canonicalFF\-AUC falls from\.668\.668to\.483\.483, the observed aggregate audit rises from\.470\.470to\.725\.725\. Support depletion is equally directional\. With correctFFlabels, canonicalFF\-AUC is\.545\.545when only eight own failures remain; adding 80 random development examples barely changes it \(\.550\.550\), whereas adding 80 targeted own failures raises it to\.606\.606\(full support\.679\.679\)\. The pair choice, leakage sweep, random\-vs\-targeted replay and the original near\-boundary disapproval control are all post hoc and reported in full in Appendix[AD\.6](https://arxiv.org/html/2610.29029#A30.SS6); AG News supplies an independent constructed replication \(Appendix[AD\.7](https://arxiv.org/html/2610.29029#A30.SS7)\)\.

Immediately before this study, the post\-DPO ECG pools already exhibit the predicted support shift: the own\-failure bin of the deployability head is empty in every evaluable pool of the four Llama gate/inference runs and in6464–72%72\\%of evaluable pools of the six Phi runs, versus13%13\\%and36%36\\%at base\. We therefore test not merely whether the monitor changes, but whether acting on it causes measurable off\-policy and downstream harm\.

#### Protocol: held\-out subjects, two refresh classes, and an unseen generator\.

We use the frozen ECG pools, whereFFis a hard deployment constraint: a valid pipeline within latency, cycle and RAM budgets\. For Llama and Phi, a specialistSSis trained on the base\-policy train pool withFFlabels, in two head classes: an MLP with the frozen critic’s architecture and gradient\-boosted trees\. Refresh data are post\-DPO test pools from one half of test subjects; every reported refresh metric is evaluated on the complementary subject half over1010random subject splits\.RyR\_\{y\}refreshes with outcome labels,RFR\_\{F\}with component labels, andRmixR\_\{\\rm mix\}with component labels plus an equal number of re\-injected base\-policy candidates\. We evaluate four levels of evidence: the on\-policy monitor, off\-policyFF\-AUC on the source generator, transferFF\-AUC on unseen Gemma outputs, and a best\-of\-1616gated consumer\. Six predictions were hashed before the run \(Appendix[AD](https://arxiv.org/html/2610.29029#A30)\); an exploratory pilot using a different head procedure and split seed had informed P1–P3, while P4–P6 were untested\. The four family×\\timeshead\-class configurations are the primary experimental settings; the 10 overlapping subject splits within each setting are robustness resamples, so counts such as40/4040/40below report sign consistency rather than 40 independent replications\.

Table 3:Refreshing a component verifier on on\-policy ECG data \(subject\-held\-out half; mean of1010paired splits; each cell MLP / GBM\)\. The monitor is full\-target on\-policy AUC\.FF\-AUC is measured off\-policy on base pools from the source generator and unseen Gemma\. The gate column tests whether verifier damage changes a downstream best\-of\-16 decision; violations up to about\.15\.15are unavoidable because some pools contain no feasible candidate\.Figure 2:\(a\) The audit prefers entanglement\.Heads trained on base data with labels\(1−λ\)​F\+λ​y\(1\-\\lambda\)F\+\\lambda y: the pure component head \(λ=0\\lambda=0\), which has canonicalFF\-AUC1\.0001\.000, receives the lowest on\-policy audit AUC in all four family×\\timesclass settings \(P1\)\.\(b\) Acting on the audit damages the component\.Refresh minus deployed specialist in each of1010held\-out\-subject splits and two head classes\. Outcome refresh lowers off\-policy canonicalFF\-AUC in every paired split while its effect on the monitor changes sign across settings; restoring base failures leaves canonicalFF\-AUC essentially unchanged\.
#### Result 1: the monitor selects the wrong representation\.

P1 held in all four family×\\timesclass settings\. Among heads trained from the same base data with label mixture\(1−λ\)​F\+λ​y\(1\-\\lambda\)F\+\\lambda y, the pure component head \(λ=0\\lambda=0\) had the*lowest*on\-policy aggregate AUC, despiteFF\-AUC1\.0001\.000\. This is the empirical analogue of Proposition[2](https://arxiv.org/html/2610.29029#Thmproposition2): adding outcome information improves the audit score while moving the representation away from a pure component verifier\.

#### Result 2: outcome refresh causes systematic off\-policy forgetting\.

P3 and P5 held\. Relative to the deployed specialist,RyR\_\{y\}lowered source\-generator off\-policyFF\-AUC with the same sign in every held\-out\-subject split across both families and both head classes \(40/4040/40paired evaluations; sign robustness, not independent replications\): by0\.090\.09–0\.120\.12for MLPs and0\.430\.43–0\.480\.48for GBMs\. The effect is not confined to the refresh generator: on unseen Gemma it drops0\.290\.29–0\.510\.51\. By contrast, the monitor itself is not a consistent indicator of this damage: its mean change is positive in two settings and negative in two, and the preregistered P2 prediction that Phi’s monitor would improve by at least0\.050\.05in both classes failed \(\+0\.042\+0\.042,−0\.012\-0\.012\)\. The intervention can therefore destroy the component without producing a reliable improvement even in the statistic that motivated it\.

#### Result 3: the forgotten component changes downstream decisions\.

P6 held\. UsingRyR\_\{y\}as the feasibility gate of a best\-of\-1616selector increases constraint violations with the same sign in every held\-out\-subject split across both families and both head classes \(40/4040/40paired evaluations; sign robustness, not independent replications\), by\+0\.011\+0\.011to\+0\.055\+0\.055above the unavoidable floor, and lowers Answer@16 by0\.020\.02–0\.060\.06\. This closes the loop from an apparently bad audit to an operationally worse consumer: the effect is not only a metric disagreement\.

#### Result 4: correct labels are insufficient; restored support is sufficient here\.

RFR\_\{F\}isolates support mismatch from label mismatch\. It still loses0\.2350\.235source\-generatorFF\-AUC in the Llama\-GBM setting and admits more violations, despite being refreshed with the correct component labels\. In contrast, P4 held forRmixR\_\{\\rm mix\}: re\-injecting base\-policy candidates while keeping component labels preservesFF\-AUC at1\.0001\.000in every source setting, keeps unseen\-generator AUC at0\.9980\.998–1\.0001\.000, and returns the gate metrics to the deployed specialist\. The difference betweenRFR\_\{F\}andRmixR\_\{\\rm mix\}is support, not supervision\.

#### Closed\-loop counterfactual\.

Composing the frozen trigger signal with the already\-trained refresh heads closes the loop without new training\. Refreshing withRyR\_\{y\}whenever aggregate audit AUC falls by at least\.05\.05from base triggers all four primary settings even though each deployed specialist retainsFF\-AUC1\.0001\.000; the resulting meanFF\-AUC/violation/Answer@16 changes from1\.000/\.148/\.5301\.000/\.148/\.530to\.719/\.179/\.491\.719/\.179/\.491\. Thresholds\.10\.10–\.25\.25trigger fewer settings but remain worse than no refresh on all three outcomes; a reference\-distributionFF\-specific trigger fires in none\. The full trigger–harm curve is shown in Appendix[AD\.5](https://arxiv.org/html/2610.29029#A30.SS5)\.

#### Empirical conclusion\.

Across the study, five of six registered predictions held\. More importantly, the evidence follows the mechanism at four levels: the audit rewards entanglement, optimization removes own\-failure support, refresh causes held\-out and cross\-generator component forgetting, and that forgetting raises downstream violations\. The failed prediction is informative rather than cosmetic: the monitored AUC is not reliably repaired by refresh, strengthening the narrower conclusion that it is an unsuitable control signal\. The successful rescue identifies the actionable fix used in the next section: preserve or deliberately reconstruct the component’s own failure distribution\.

## 6Repair: restore failure support

#### When can the audit recover component behavior?

Three routes exist: condition directly on semantic failure bins when they have support; impose an externally justified restrictionAcross∈\[ℓ,u\]A\_\{\\rm cross\}\\in\[\\ell,u\], which forq<1q<1gives

Aown∈\[max⁡\(0,A−q​u1−q\),min⁡\(1,A−q​ℓ1−q\)\];A\_\{\\rm own\}\\in\\left\[\\max\\\!\\left\(0,\\tfrac\{A\-qu\}\{1\-q\}\\right\),\\min\\\!\\left\(1,\\tfrac\{A\-q\\ell\}\{1\-q\}\\right\)\\right\];\(6\)or vary the failure mixture with bin\-conditional distributions fixed, which by Proposition[1](https://arxiv.org/html/2610.29029#Thmproposition1)reduces recovery to the rank of the mixture matrix and also restores support lost to saturation \(Appendix[P](https://arxiv.org/html/2610.29029#A16)\)\.This is a designed reweighting/resampling operation, not a license to treat arbitrary policy checkpoints as mixture rows:if either bin\-conditional score distribution changes, naive two\-row inversion is biased\. Appendix[S](https://arxiv.org/html/2610.29029#A19)gives the explicit drift term and an empirical boundary case whereAownA\_\{\\rm own\}shifts from\.965\.965to\.821\.821out of distribution and the transported forecast misses by\.101\.101\. Across two policies, the AUC change splits exactly into a mixture\-shift term and conditional\-ordering drift, so both mechanisms can coexist \(Eq\.[7](https://arxiv.org/html/2610.29029#A29.E7), Appendix[AC](https://arxiv.org/html/2610.29029#A29)\)\.

#### An audit card\.

When aggregate outcome AUC changes for a component head:\(1\)define semantic failure bins and report their weights and per\-pool identified intervals \(Eq\.[4](https://arxiv.org/html/2610.29029#S4.E4)\) with a cluster\-bootstrap outer interval, flagging pools withq≥max⁡\(A,1−A\)q\\geq\\max\(A,1\-A\);\(2\)if the interval is uninformative, labelmmrandom off\-target candidates and estimate the pair\-weighted within\-poolAownA\_\{\\rm own\}directly;\(3\)if the own\-failure bin lacks support, construct two controlled negative mixtures by reweighting or resampling fixed bin\-conditional candidates, then invert the2×22\\times 2system of Eq\.[3](https://arxiv.org/html/2610.29029#S4.E3);\(4\)do not use aggregate on\-policy outcome AUC as the sole criterion to select, early\-stop or refresh a component verifier \(Proposition[2](https://arxiv.org/html/2610.29029#Thmproposition2)\); refresh only on data that contain its own failures and report its quality off\-policy \(Section[5](https://arxiv.org/html/2610.29029#S5)\);\(5\)audit separately the interface that consumes the score \(Appendices[AE](https://arxiv.org/html/2610.29029#A31)–[AF](https://arxiv.org/html/2610.29029#A32)\)\. The next result prices steps 2 and 3\.

###### Proposition 3\(Cost of restoring identification\)\.

Fix a pool’s positives and leth⁡\(x\)∈\[0,1\]h\(x\)\\in\[0,1\]be the fraction of them the head ranks above a negativexx\(ties12\\tfrac\{1\}\{2\}\), soAown=𝔼⁡\[h⁡\(X\)∣X∈Bown\]A\_\{\\rm own\}=\\mathbb\{E\}\[h\(X\)\\mid X\\in B\_\{\\rm own\}\]\.\(i\) Passive audit\.Labelmmnegatives drawn i\.i\.d\. from the pool’s failure mixture and averagehhover the revealed own failures\. Ifm⁡\(1−q\)≥max⁡\{8​log⁡\(2/δ\),log⁡\(4/δ\)/ε2\}m\(1\-q\)\\geq\\max\\\{8\\log\(2/\\delta\),\\,\\log\(4/\\delta\)/\\varepsilon^\{2\}\\\}, the estimate is withinε\\varepsilonofAownA\_\{\\rm own\}with probability at least1−δ1\-\\delta\. Conversely, forε≤14\\varepsilon\\leq\\tfrac\{1\}\{4\}andδ<14\\delta<\\tfrac\{1\}\{4\}, every estimator built from such a sample errs by at leastε\\varepsilonwith probability aboveδ\\deltaon some instance unlessm⁡\(1−q\)≥max⁡\{q​log⁡\(1/\(2​δ\)\),log⁡\(1/\(4​δ\)\)/\(24​ε2\)\}m\(1\-q\)\\geq\\max\\\{q\\log\(1/\(2\\delta\)\),\\,\\log\(1/\(4\\delta\)\)/\(24\\varepsilon^\{2\}\)\\\}, so the two budgets match up to constants\.\(ii\) Designed audit\.If aggregate AUCs are estimated withinε1,ε2\\varepsilon\_\{1\},\\varepsilon\_\{2\}at known weightsq1≠q2q\_\{1\}\\neq q\_\{2\}, inverting Eq\.[3](https://arxiv.org/html/2610.29029#S4.E3)gives\|A^own−Aown\|≤\(q2​ε1\+q1​ε2\)/\|q2−q1\|\|\\hat\{A\}\_\{\\rm own\}\-A\_\{\\rm own\}\|\\leq\(q\_\{2\}\\varepsilon\_\{1\}\+q\_\{1\}\\varepsilon\_\{2\}\)/\|q\_\{2\}\-q\_\{1\}\|, independent of the deployed policy’sqq\.

The passive audit thus becomes expensive exactly where Proposition[1](https://arxiv.org/html/2610.29029#Thmproposition1)makes the aggregate uninformative, and impossible once the own bin is empty; a designed mixture removes the dependence \(proofs in Appendix[AA](https://arxiv.org/html/2610.29029#A27)\)\. On the ECG utility head,m⁡\(1−q\)⋅RMSE\\sqrt\{m\(1\-q\)\}\\cdot\\mathrm\{RMSE\}stays at0\.200\.20–0\.240\.24for everyq≤0\.95q\\leq 0\.95,m=100m=100labels give9696–98%98\\%subject\-bootstrap coverage, and two\-mixture inversion reaches RMSE0\.010\.01–0\.020\.02at\|q2−q1\|=0\.6\|q\_\{2\}\-q\_\{1\}\|=0\.6\(Appendices[Y](https://arxiv.org/html/2610.29029#A25),[AA](https://arxiv.org/html/2610.29029#A27)\)\. The same designed mixture is what protects a refresh:RmixR\_\{\\rm mix\}in Table[3](https://arxiv.org/html/2610.29029#S5.T3)differs fromRFR\_\{F\}only by re\-injected base\-policy failures and keeps canonicalFF\-AUC at1\.0001\.000in every setting, and re\-injection likewise recoversAownA\_\{\\rm own\}with RMSE≤0\.02\\leq 0\.02on post\-DPO ECG pools whose own bin is empty \(Appendix[AA](https://arxiv.org/html/2610.29029#A27)\)\.

## 7Scope: what the results do and do not establish

The AUC symptom does not identify its cause: an exact oracle collapses and an unbounded autoencoder genuinely exploits the critic \(feasibility0\.99990\.9999predicted,1\.8%1\.8\\%actual; Appendix[F](https://arxiv.org/html/2610.29029#A6)\); this is not a claim that in\-bounds exploitation is absent\. Proposition[2](https://arxiv.org/html/2610.29029#Thmproposition2)is exact, but its engine and GSM8K checks reuse published quantities and are post\-hoc\. The GoEmotions study is a post\-hoc natural multi\-label text stress test with a small bag\-of\-words verifier; the AG News experiment is a more constructed control with 24 deterministic robustness splits rather than independent replications\. Together they establish that the mechanism is not specific to engineered numeric features, not that our refresh effect has been measured with an LLM\-scale semantic verifier\. Contemporary rubric\-RL work independently shows criterion\-level saturation and scalar aggregation at LLM scale\([Gunjal et al\., 2026](https://arxiv.org/html/2610.29029#bib.bib42);[Tyagi et al\., 2026](https://arxiv.org/html/2610.29029#bib.bib43)\); we use that literature only to establish interface relevance, not as validation of our inversion or refresh claims\. The ECG refresh study is CPU\-only and uses one benchmark whose component is a deterministic resource constraint, two head classes and ten subject splits; we did not refresh an LLM\-scale verifier, did not study online RL with a refreshed verifier, and our prediction that refresh repairs the monitor failed\. Engine and GSM8K policies are Qwen2\.5, the strongest training result is offline DPO with three seeds, and ECG training is informative in one family\. The audit requires trusted component labels \(proxy bins change the estimand\); the engine oracle is simplified, and ECG is a signal\-processing benchmark, not a clinical validation\.

## 8Related work

Modern post\-training makes the component/aggregate interface concrete\. Rubric RL scores prompt\-specific criteria separately and then aggregates them into scalar on\-policy rewards\([Gunjal et al\., 2026](https://arxiv.org/html/2610.29029#bib.bib42);[Mahmoud et al\., 2026](https://arxiv.org/html/2610.29029#bib.bib5)\); policy\-aware rubric work further shows that criterion informativeness changes with the policy because many criteria become saturated or dead\([Tyagi et al\., 2026](https://arxiv.org/html/2610.29029#bib.bib43)\)\. Fresh policy interactions are also used to refine reward or preference models\([Sun et al\., 2025](https://arxiv.org/html/2610.29029#bib.bib12);[Bai et al\., 2022](https://arxiv.org/html/2610.29029#bib.bib11)\)\. Two adjacent loop failures are especially relevant: on\-policy optimization against one verifier can remove behaviors needed for later objectives\([Wei et al\., 2026](https://arxiv.org/html/2610.29029#bib.bib21)\), and a verifier useful on one task can regress a learner after task shift\([Lin, 2026](https://arxiv.org/html/2610.29029#bib.bib20)\)\. We instead study the*audit interface*: full\-outcome AUC can stop identifying one component, prefer an entangled verifier, and induce a harmful refresh even when the component verifier was initially correct\.

The statistical ingredients are classical\. ROC behavior depends on case mix\([Sadatsafavi et al\., 2022](https://arxiv.org/html/2610.29029#bib.bib22)\); partial identification and trimming bounds describe incomplete observables\([Manski, 2003](https://arxiv.org/html/2610.29029#bib.bib23);[Tamer, 2010](https://arxiv.org/html/2610.29029#bib.bib24);[Horowitz and Manski, 1995](https://arxiv.org/html/2610.29029#bib.bib15)\); deliberately varying mixture weights is a linear special case of finite\-mixture identification\([Henry et al\., 2014](https://arxiv.org/html/2610.29029#bib.bib25)\); the within/cross split parallels xAUC\([Kallus and Zhou, 2019](https://arxiv.org/html/2610.29029#bib.bib16)\); and the population AUC maximizer is the classical optimal ranker\([Clémençon et al\., 2008](https://arxiv.org/html/2610.29029#bib.bib13)\)\. We do not claim these ingredients as new\. The new consequence is their combination under policy\-induced*semantic*failure mixtures: complete loss of component identification, a calculable verifier\-ranking inversion, and an audit\-driven maintenance failure\.

## AI use statement

Generative AI was used to improve the clarity and presentation of manuscript claims, assist with software implementation and debugging, and provide critical feedback on claims and arguments during manuscript development\. AI\-generated suggestions were treated as editorial and technical assistance rather than as evidence or independent validation\. The authors are fully responsible for the final claims, proofs, code, citations, analyses, and submission\.

## Ethics statement

The engine evaluations concern a simplified geometry, mass and printability model, not safety certification or fabrication validation; propulsion design can have dual\-use implications\. The ECG study uses public, de\-identified PhysioNet recordings \(MIT\-BIH Arrhythmia and Noise Stress Test databases\) under their published terms; it is a signal\-processing deployment benchmark, not a diagnostic or clinical validation, and its outputs should not be used for patient care\.

## Reproducibility statement

The supplement contains the exact anonymous source snapshot, the CPU\-only GoEmotions and AG News semantic stress\-test scripts with frozen JSON summaries, and the frozen controller threshold analysis used in the final revision\. The original large engine/ECG/GSM8K candidate pools and model checkpoints are not bundled in this lightweight artifact; their protocols, registered predictions, assumptions and reported frozen summaries are documented in the appendix\. Thus the supplement supports an exact paper rebuild and independent reruns of the public\-data semantic controls, but not an end\-to\-end regeneration of every large system experiment from raw candidates\. See Appendices[G](https://arxiv.org/html/2610.29029#A7),[AD\.6](https://arxiv.org/html/2610.29029#A30.SS6),[AD\.7](https://arxiv.org/html/2610.29029#A30.SS7),[I](https://arxiv.org/html/2610.29029#A9)and[K](https://arxiv.org/html/2610.29029#A11)\.

## References

- Baiet al\.\(2026\)Y\. Bai, K\. Liu, Z\. Zhuang, J\. Zhou, R\. Weng, X\. Chen, J\. Wang, and X\. CaiMulti\-objective and mixed\-reward reinforcement learning via reward\-decorrelated policy optimization\.arXiv preprint arXiv:2605\.13641\.External Links:[Link](https://arxiv.org/abs/2605.13641)Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.12.1.1.1),[§1](https://arxiv.org/html/2610.29029#S1.p1.1)\.
- Baiet al\.\(2022\)Y\. Bai, A\. Jones, K\. Ndousse, A\. Askell, A\. Chen, N\. DasSarma, D\. Drain, S\. Fort, D\. Ganguli, T\. Henighan,et al\.Training a helpful and harmless assistant with reinforcement learning from human feedback\.arXiv preprint arXiv:2204\.05862\.Cited by:[Table 34](https://arxiv.org/html/2610.29029#A30.T34.2.5.1),[§1](https://arxiv.org/html/2610.29029#S1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.29029#S1.p1.1),[§8](https://arxiv.org/html/2610.29029#S8.p1.1)\.
- Bentleyet al\.\(1978\)J\. L\. Bentley, H\. T\. Kung, M\. Schkolnick, and C\. D\. ThompsonOn the average number of maxima in a set of vectors and applications\.Journal of the ACM25\(4\),pp\. 536–543\.Cited by:[§AA\.2](https://arxiv.org/html/2610.29029#A27.SS2.p1.1),[Appendix AE](https://arxiv.org/html/2610.29029#A31.p3.1)\.
- Brownet al\.\(2024\)B\. Brown, J\. Juravsky, R\. Ehrlich, R\. Clark, Q\. V\. Le, C\. Ré, and A\. MirhoseiniLarge language monkeys: scaling inference compute with repeated sampling\.arXiv preprint arXiv:2407\.21787\.Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.10.1.1.1),[§1](https://arxiv.org/html/2610.29029#S1.p1.1)\.
- Clémençonet al\.\(2008\)S\. Clémençon, G\. Lugosi, and N\. VayatisRanking and empirical minimization of U\-statistics\.The Annals of Statistics36\(2\),pp\. 844–874\.Cited by:[§AD\.1](https://arxiv.org/html/2610.29029#A30.SS1.p1.1),[§4\.1](https://arxiv.org/html/2610.29029#S4.SS1.p2.1),[§8](https://arxiv.org/html/2610.29029#S8.p2.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.10.1.1.1),[§1](https://arxiv.org/html/2610.29029#S1.p1.1)\.
- Daiet al\.\(2024\)J\. Dai, X\. Pan, R\. Sun, J\. Ji, X\. Xu, M\. Liu, Y\. Wang, and Y\. YangSafe RLHF: safe reinforcement learning from human feedback\.InInternational Conference on Learning Representations,Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.12.1.1.1)\.
- Demszkyet al\.\(2020\)D\. Demszky, D\. Movshovitz\-Attias, J\. Ko, A\. Cowen, G\. Nemade, and S\. RaviGoEmotions: a dataset of fine\-grained emotions\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,pp\. 4040–4054\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.372),[Link](https://aclanthology.org/2020.acl-main.372/)Cited by:[§AD\.6](https://arxiv.org/html/2610.29029#A30.SS6.p1.1),[§5](https://arxiv.org/html/2610.29029#S5.SS0.SSS0.Px1.p1.1)\.
- Dorneret al\.\(2026\)F\. E\. Dorner, Y\. Chen, A\. F\. Cruz, and F\. YangROC\-n\-reroll: how verifier imperfection affects test\-time scaling\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=3Gy5mmyuxn)Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.10.1.1.1),[Appendix R](https://arxiv.org/html/2610.29029#A18.p2.1)\.
- Gaoet al\.\(2023\)L\. Gao, J\. Schulman, and J\. HiltonScaling laws for reward model overoptimization\.InInternational Conference on Machine Learning,Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.9.1.1.1)\.
- Gunjalet al\.\(2026\)A\. Gunjal, A\. Wang, E\. Lau, V\. Nath, Y\. He, B\. Liu, and S\. HendryxRubrics as rewards: reinforcement learning beyond verifiable domains\.InInternational Conference on Learning Representations,Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.4.1.1.1),[Table 34](https://arxiv.org/html/2610.29029#A30.T34.2.2.1),[§1](https://arxiv.org/html/2610.29029#S1.SS0.SSS0.Px2.p1.1),[§7](https://arxiv.org/html/2610.29029#S7.p1.1),[§8](https://arxiv.org/html/2610.29029#S8.p1.1)\.
- Hardtet al\.\(2016\)M\. Hardt, B\. Recht, and Y\. SingerTrain faster, generalize better: stability of stochastic gradient descent\.InInternational Conference on Machine Learning,Cited by:[§AA\.3](https://arxiv.org/html/2610.29029#A27.SS3.p1.1),[Appendix AF](https://arxiv.org/html/2610.29029#A32.p2.1)\.
- Henryet al\.\(2014\)M\. Henry, Y\. Kitamura, and B\. SalaniéPartial identification of finite mixtures in econometric models\.Quantitative Economics5\(1\),pp\. 123–144\.External Links:[Document](https://dx.doi.org/10.3982/QE170)Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.3.1.1.1),[§8](https://arxiv.org/html/2610.29029#S8.p2.1)\.
- Hoeffding \(1963\)W\. HoeffdingProbability inequalities for sums of bounded random variables\.Journal of the American Statistical Association58\(301\),pp\. 13–30\.Cited by:[§AA\.1](https://arxiv.org/html/2610.29029#A27.SS1.p1.1)\.
- Horowitz and Manski \(1995\)J\. L\. Horowitz and C\. F\. ManskiIdentification and robustness with contaminated and corrupted data\.Econometrica63\(2\),pp\. 281–302\.Cited by:[§AB\.1](https://arxiv.org/html/2610.29029#A28.SS1.p1.2),[§8](https://arxiv.org/html/2610.29029#S8.p2.1),[Proposition 1](https://arxiv.org/html/2610.29029#Thmproposition1.p1.4)\.
- Kallus and Zhou \(2019\)N\. Kallus and A\. ZhouThe fairness of risk scores beyond classification: bipartite ranking and the xAUC metric\.InAdvances in Neural Information Processing Systems,Cited by:[§4\.1](https://arxiv.org/html/2610.29029#S4.SS1.p2.1),[§8](https://arxiv.org/html/2610.29029#S8.p2.1)\.
- Kimet al\.\(2025\)S\. Kim, D\. Kang, T\. Kwon, H\. Chae, D\. Lee, and J\. YeoRethinking reward model evaluation through the lens of reward overoptimization\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 13252–13280\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.649),[Link](https://aclanthology.org/2025.acl-long.649/)Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.5.1.1.1)\.
- Kirkpatricket al\.\(2017\)J\. Kirkpatrick, R\. Pascanu, N\. Rabinowitz, J\. Veness, G\. Desjardins, A\. A\. Rusu, K\. Milan, J\. Quan, T\. Ramalho, A\. Grabska\-Barwinska, D\. Hassabis, C\. Clopath, D\. Kumaran, and R\. HadsellOvercoming catastrophic forgetting in neural networks\.Proceedings of the National Academy of Sciences114\(13\),pp\. 3521–3526\.Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.8.1.1.1)\.
- Lightmanet al\.\(2024\)H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. CobbeLet’s verify step by step\.InInternational Conference on Learning Representations,Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.10.1.1.1),[§1](https://arxiv.org/html/2610.29029#S1.p1.1)\.
- Lin \(2026\)J\. LinWhen good verifiers go bad: silent negative transfer in verifier\-guided VLM training\.arXiv preprint arXiv:2606\.14629\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2606.14629),[Link](https://arxiv.org/abs/2606.14629)Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.7.1.1.1),[§8](https://arxiv.org/html/2610.29029#S8.p1.1)\.
- Liuet al\.\(2026\)S\. Liu, X\. Dong, X\. Lu, S\. Diao,et al\.GDPO: group reward\-decoupled normalization policy optimization for multi\-reward RL optimization\.arXiv preprint arXiv:2601\.05242\.External Links:[Link](https://arxiv.org/abs/2601.05242)Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.12.1.1.1),[§1](https://arxiv.org/html/2610.29029#S1.p1.1)\.
- Mahmoudet al\.\(2026\)A\. Mahmoud, M\. Rezaei, Z\. Wang, A\. Gunjal, B\. Liu, and Y\. HeReward hacking in rubric\-based reinforcement learning\.arXiv preprint arXiv:2605\.12474\.External Links:[Link](https://arxiv.org/abs/2605.12474)Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.9.1.1.1),[Table 34](https://arxiv.org/html/2610.29029#A30.T34.2.3.1),[§1](https://arxiv.org/html/2610.29029#S1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.29029#S1.p1.1),[§8](https://arxiv.org/html/2610.29029#S8.p1.1)\.
- Manski \(2003\)C\. F\. ManskiPartial identification of probability distributions\.Springer\.Cited by:[§8](https://arxiv.org/html/2610.29029#S8.p2.1)\.
- Miettinen \(1999\)K\. MiettinenNonlinear multiobjective optimization\.Kluwer Academic Publishers\.Cited by:[Appendix AE](https://arxiv.org/html/2610.29029#A31.p2.1)\.
- Moskovitzet al\.\(2024\)T\. Moskovitz, A\. K\. Singh, D\. Strouse, T\. Sandholm, R\. Salakhutdinov, A\. D\. Dragan, and S\. McAleerConfronting reward model overoptimization with constrained RLHF\.InInternational Conference on Learning Representations,Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.12.1.1.1),[§1](https://arxiv.org/html/2610.29029#S1.p1.1)\.
- Rafailovet al\.\(2024\)R\. Rafailov, Y\. Chittepu, R\. Park, H\. Sikchi, J\. Hejna, W\. B\. Knox, C\. Finn, and S\. NiekumScaling laws for reward model overoptimization in direct alignment algorithms\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 126207–126242\.External Links:[Document](https://dx.doi.org/10.52202/079017-4009)Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.9.1.1.1)\.
- Raméet al\.\(2023\)A\. Ramé, G\. Couairon, C\. Dancette, J\. Gaya, M\. Shukor, L\. Soulier, and M\. CordRewarded soups: towards Pareto\-optimal alignment by interpolating weights fine\-tuned on diverse rewards\.InAdvances in Neural Information Processing Systems,Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.12.1.1.1)\.
- Razinet al\.\(2025\)N\. Razin, Z\. Wang, H\. Strauss, S\. Wei, J\. D\. Lee, and S\. AroraWhat makes a reward model a good teacher? an optimization perspective\.InAdvances in Neural Information Processing Systems,External Links:[Document](https://dx.doi.org/10.52202/085713-1981)Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.5.1.1.1)\.
- Robins \(1995\)A\. RobinsCatastrophic forgetting, rehearsal and pseudorehearsal\.Connection Science7\(2\),pp\. 123–146\.Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.8.1.1.1)\.
- Romanoet al\.\(2026\)D\. Romano, K\. Raj, J\. Parker, and D\. GiofrèTest\-time scaling in the wild: why exploitation, not exploration, is the bottleneck\.arXiv preprint arXiv:2608\.18931\.Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.10.1.1.1)\.
- Sadatsafaviet al\.\(2022\)M\. Sadatsafavi, P\. Saha\-Chaudhuri, and J\. PetkauModel\-based roc curve: examining the effect of case mix and model calibration on the roc plot\.Medical Decision Making42\(4\),pp\. 487–499\.External Links:[Document](https://dx.doi.org/10.1177/0272989X211050909)Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.2.1.1.1),[§8](https://arxiv.org/html/2610.29029#S8.p2.1)\.
- Shumet al\.\(2026\)K\. Shum, B\. Hui, J\. Chen, L\. Zhang, X\. W\., J\. Yang, Y\. Huang, J\. Lin, and J\. HeSWE\-RM: execution\-free feedback for software engineering agents\.InInternational Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2512.21919)Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.11.1.1.1),[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.5.1.1.1)\.
- Skalseet al\.\(2022\)J\. Skalse, N\. H\. R\. Howe, D\. Krasheninnikov, and D\. KruegerDefining and characterizing reward hacking\.InAdvances in Neural Information Processing Systems,Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.9.1.1.1)\.
- Snellet al\.\(2025\)C\. Snell, J\. Lee, K\. Xu, and A\. KumarScaling LLM test\-time compute optimally can be more effective than scaling parameters for reasoning\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=4FWaWZtd2n)Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.10.1.1.1),[§1](https://arxiv.org/html/2610.29029#S1.p1.1)\.
- Sunet al\.\(2025\)Z\. Sun, Y\. Guo, Y\. Lin, X\. Chen, Q\. Qi, X\. Tang, X\. He, and J\. WenUncertainty and influence aware reward model refinement for reinforcement learning from human feedback\.InInternational Conference on Learning Representations,Cited by:[Table 34](https://arxiv.org/html/2610.29029#A30.T34.2.5.1),[§1](https://arxiv.org/html/2610.29029#S1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.29029#S1.p1.1),[§8](https://arxiv.org/html/2610.29029#S8.p1.1)\.
- Tamer \(2010\)E\. TamerPartial identification in econometrics\.Annual Review of Economics2,pp\. 167–195\.Cited by:[§8](https://arxiv.org/html/2610.29029#S8.p2.1)\.
- Tsybakov \(2009\)A\. B\. TsybakovIntroduction to nonparametric estimation\.Springer\.Cited by:[§AA\.1](https://arxiv.org/html/2610.29029#A27.SS1.p1.1)\.
- Tyagiet al\.\(2026\)U\. Tyagi, X\. Guo, M\. Rezaei, D\. George, A\. Mahmoud, J\. Lee, B\. Liu, and Y\. HeNot every rubric teaches equally: policy\-aware rubric rewards for RLVR\.arXiv preprint arXiv:2605\.20164\.Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.4.1.1.1),[§AD\.3](https://arxiv.org/html/2610.29029#A30.SS3.p1.1),[Table 34](https://arxiv.org/html/2610.29029#A30.T34.2.4.1),[§1](https://arxiv.org/html/2610.29029#S1.SS0.SSS0.Px2.p1.1),[§7](https://arxiv.org/html/2610.29029#S7.p1.1),[§8](https://arxiv.org/html/2610.29029#S8.p1.1)\.
- Weiet al\.\(2026\)S\. Wei, Z\. Su, F\. Song, W\. Luo, W\. Li, G\. Peng, and H\. WangVerifier\-induced support reshaping in on\-policy optimization\.arXiv preprint arXiv:2608\.00220\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2608.00220),[Link](https://arxiv.org/abs/2608.00220)Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.6.1.1.1),[§8](https://arxiv.org/html/2610.29029#S8.p1.1)\.
- Wenet al\.\(2025\)X\. Wen, J\. Lou, Y\. Lu, H\. Lin, X\. Yu, X\. Lu, B\. He, X\. Han, D\. Zhang, and L\. SunRethinking reward model evaluation: are we barking up the wrong tree?\.InInternational Conference on Learning Representations,Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.5.1.1.1)\.
- Wuet al\.\(2023\)Z\. Wu, Y\. Hu, W\. Shi, N\. Dziri, A\. Suhr, P\. Ammanabrolu, N\. A\. Smith, M\. Ostendorf, and H\. HajishirziFine\-grained human feedback gives better rewards for language model training\.InAdvances in Neural Information Processing Systems,Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.12.1.1.1),[§1](https://arxiv.org/html/2610.29029#S1.p1.1)\.
- Zhanget al\.\(2015\)X\. Zhang, J\. Zhao, and Y\. LeCunCharacter\-level convolutional networks for text classification\.InAdvances in Neural Information Processing Systems,pp\. 649–657\.Cited by:[§AD\.7](https://arxiv.org/html/2610.29029#A30.SS7.p1.1)\.
- Zhouet al\.\(2024\)Z\. Zhou, J\. Liu, J\. Shao, X\. Yue, C\. Yang, W\. Ouyang, and Y\. QiaoBeyond one\-preference\-fits\-all alignment: multi\-objective direct preference optimization\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 10586–10613\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.630),[Link](https://aclanthology.org/2024.findings-acl.630/)Cited by:[Table 9](https://arxiv.org/html/2610.29029#A10.T9.2.12.1.1.1),[§1](https://arxiv.org/html/2610.29029#S1.p1.1)\.

## Appendix roadmap

The appendix is intentionally evidence\-first\. Appendices A–AD contain the oracle definitions, core audit\-collapse controls, identification proofs, transfer checks, preregistration material and the refresh/semantic experiments that support the main claim\. Appendices AE–AF are optional boundary analyses of downstream selection and DPO interfaces; they are not needed for Propositions 1–3 or the refresh\-trap conclusion\. Bundled scripts and frozen summaries are indexed in the supplement README; appendix\-only workspace paths are provenance labels and do not imply that the corresponding large artifacts are packaged\.

## Appendix AThe engine oracle

The solver decodes a design vector to a multi\-zone body, computes its mass by integration, evaluates printability \(overhang, minimum ligament and wall, DfAM flags\) and envelope fit, and returnsfeasibleas their conjunction, in7\.97\.9ms; the200 000200\\,000\-row reference sample took7272s\. Its thermochemistry is checked against NASA CEA throughrocketcea, independent of the project’s own Cantera path: across2828operating points the maximum error is0\.25%0\.25\\%on chamber temperature and1\.7%1\.7\\%on characteristic velocity\. The flow solver is grid\-convergent on the Sod problem \(L1L\_\{1\}error0\.0079→0\.0042→0\.00240\.0079\\to 0\.0042\\to 0\.0024at100/200/400100/200/400cells\) and reproduces the oblique\-shock relation to1\.2∘1\.2^\{\\circ\}\. Those checks cover the flow path, which is sized from the requirement and not the generator’s to choose\. The on\-target criterion is two\-sided because under a one\-sided one86\.4%86\.4\\%of on\-target draws in the reference sample satisfy it by being small rather than by meeting the budget\.

## Appendix BThe identity for the mass head

With the roles swapped, the mass head’s score againsty=F∧My=F\\wedge Mdecomposes as\(1−q\)AUC\(sM;y=1vsF=0\)\+qAUC\(sM;y=1vsF=1,M=0\)\(1\-q\)\\,\\mathrm\{AUC\}\(s\_\{M\};y\{=\}1\\text\{ vs \}F\{=\}0\)\+q\\,\\mathrm\{AUC\}\(s\_\{M\};y\{=\}1\\text\{ vs \}F\{=\}1,M\{=\}0\)\. The second term is the head’s own sub\-problem and is0\.960\.96–0\.990\.99on every arm; the first is0\.840\.84–0\.950\.95, since infeasible designs are also, on average, further from the budget\. Asq→1q\\to 1the mass head’s AUC againstyyrises to its own accuracy \(0\.9370\.937on the base pool,0\.980\.98on the trained pools\)\.

## Appendix CGSM8K with a conjunctive target

Table 4:GSM8K,500500test prompts, three training seeds per trained arm\. Upper: policy rates, median within\-pool score SD atK=64K=64, and correctness\-head AUC against the conjunctive target\. Lower: supply and answer atK=16K=16\(first1616of6464samples\) under sum, soft log\-probability and correctness\-only selectors\.
AtK=64K\{=\}64supply is0\.9480\.948\(base\) and0\.9670\.967–0\.9690\.969\(trained\); best\-of\-6464by the sum is0\.7620\.762\(base\) against0\.645±0\.0120\.645\\pm 0\.012and0\.657±0\.0110\.657\\pm 0\.011\. Pairs built from the base train slice by the sum and by the gate coincide \(chosen on target0\.5210\.521against0\.5200\.520, rejected0\.0790\.079against0\.0800\.080\); pairs built from a trained policy’s own samples separate on target from off at0\.700\.70against0\.590\.59under each tested rule, which closely follows the correctness head there\.

## Appendix DEngine arms not in the main text

#### Sampling budget\.

From one6464\-sample run per arm \(firstKKtaken, nested\), answer rate under the two\-stage gate, one seed each \(1 6001\\,600\-pair policies\): 1\.5B base0\.028/0\.078/0\.204/0\.291/0\.3910\.028/0\.078/0\.204/0\.291/0\.391and 1\.5B DPO0\.104/0\.265/0\.452/0\.512/0\.5610\.104/0\.265/0\.452/0\.512/0\.561atK=1/4/16/32/64K=1/4/16/32/64; 7B base0\.044/0\.142/0\.318/0\.441/0\.5440\.044/0\.142/0\.318/0\.441/0\.544and 7B DPO0\.096/0\.204/0\.365/0\.428/0\.5040\.096/0\.204/0\.365/0\.428/0\.504\. The single\-seed 7B reversal atK=64K\{=\}64is1\.81\.8standard errors and uses the weakest of the three 7B seeds; we do not claim it\.

#### Training\-side mitigations\.

DPO atβ∈\{0\.1,0\.5,1\.0\}\\beta\\in\\\{0\.1,0\.5,1\.0\\\}and IPO at target margins0\.0050\.005and0\.0200\.020on the 1\.5B designer \(1 6001\\,600pairs, one seed each\) reach feasible rates0\.390\.39–0\.870\.87; each sits on Eq\.[3](https://arxiv.org/html/2610.29029#S4.E3)at its ownqq\(Table[2](https://arxiv.org/html/2610.29029#S4.T2)\); under the two\-stage gate they capture0\.750\.75–0\.930\.93of their supply and the best composed system is the one with the most supply \(β=1\.0\\beta=1\.0,0\.5140\.514\)\. An IPO target40×40\\timesout of reach \(5\.05\.0nats/token\) broke the output format \(parse rate0\.400\.40\) and is excluded from every table\.

#### Scale\.

Under the gate the 7B designer after DPO reaches0\.414±0\.0290\.414\\pm 0\.029atK=16K\{=\}16against the 1\.5B’s0\.439±0\.0150\.439\\pm 0\.015; its feasibility head saturates to the same endpoint with five times the seed spread\. The 7B base policy’s parse rate is0\.760\.76; on parsed candidates its feasible rate is0\.430\.43\.

#### Latent search\.

Two hundred steps of gradient ascent on the unconstrained autoencoder’s latent code against the critic’s two terms give supply0\.937±0\.0170\.937\\pm 0\.017atK=64K\{=\}64against0\.336±0\.0030\.336\\pm 0\.003for direct sampling \(three decoder seeds\), not compute\-matched\. Its pools are the ones withAcross=0\.79A\_\{\\rm cross\}=0\.79: joint ascent couples the heads\.

Figure 3:Training\-side supply and selection geometry\. Left: supply againstKK\(seed means; bands one sd for packaged arms\)\. Right: per\-draw success against composed answer under the gate, one marker per seed\.

## Appendix EThe designs in parameter space

Table 5:On\-target designs of each policy \(test split, all seeds pooled\) against the prior’s on\-target designs in the reference sample\. “At bound”: radius controls within0\.050\.05of00or11; “block\-like”: every radius control above0\.90\.9; “sd”: mean per\-coordinate standard deviation;L2L\_\{2\}: mean pairwise distance between designs; “families”: share of the four lattice families \(gyroid / Schwarz\-P / diamond / IWP\)\.Every policy’s on\-target designs have2\.852\.85–2\.972\.97active zones on average, a printability score of1\.01\.0, and mass at0\.990\.99of the budget on average \(design\_degeneracy\.py\)\. The benchmark — oracle, critic, splits, pools and every result file — is released with the paper\.

#### Within\-requirement diversity\.

Conditional on at least two on\-target samples per requirement, mean within\-requirement coordinate spread is0\.1010\.101–0\.1040\.104for matched sum DPO and0\.0970\.097–0\.1030\.103for gated DPO\. It falls to0\.0380\.038–0\.0390\.039in the sum\-ranked second round, against0\.0590\.059–0\.0640\.064for gated ranking\. This conditions on successful samples and is descriptive\.

## Appendix FThe escape

![Refer to caption](https://arxiv.org/html/2610.29029v1/figures/compare_arms.png)Figure 4:One design from each cVAE arm’s pool, decoded by the function the solver grades \(longitudinal section above, cross\-section at mid\-length below\)\. The prior, the unconstrained decoder and latent search produce engines with visibly different lattice families; the decoder trained on the critic’s terms produces a block — every radius control pinned at maximum after clipping from far outside the box — that passes the mass\-and\-printability criterion and that the critic scored feasible at0\.99990\.9999before clipping\.
## Appendix GTraining details

Policies: Qwen2\.5\-1\.5B\-Instruct and Qwen2\.5\-7B\-Instruct, LoRA rank1616\(α=32\\alpha=32, dropout0\.050\.05\), DPOβ=0\.1\\beta=0\.1\(sigmoid loss\), learning rate5×10−55\\times 10^\{\-5\}, batch4×4\\timesaccumulation22at 1\.5B and2×42\\times 4at 7B, three epochs, max length25602560for engine DPO, bf16, trl\. Pairs: the score’s top\-44against bottom\-44of1616samples per requirement \(engine\) or of88per prompt \(GSM8K\); unparseable completions carry the score−103\-10^\{3\}and so form the rejected side\. Engine pair pools: the original arms used400400requirements drawn by the bench \(1 6001\\,600pairs,≈600\\approx 600steps\); the round\-1 arms of Table[39](https://arxiv.org/html/2610.29029#A32.T39)use the first1 0001\\,000requirements of the committed train split \(4 0004\\,000pairs,≈1 500\\approx 1\\,500steps\); round 2 starts from the round\-1 sum policy \(seed 0\), merged, which is also its reference\. Seeds vary initialisation and batch order; the pairs are deterministic given the pool\. GSM8K heads: RoBERTa\-base, two labels, two epochs, learning rate2×10−52\\times 10^\{\-5\}, on12 00012\\,000\(prompt, completion\) pairs from the train slice with a15%15\\%validation split\.

Engine DPO hard\-gated ranking uses the squared log\-mass proximity score forP⁡\(F\)≥0\.5P\(F\)\\geq 0\.5and subtracts100100below the threshold; unparsed rows remain at−1000\-1000\. Two\-stage evaluation instead minimises relative linear\-mass error among passing candidates and falls back to maximumP⁡\(F\)P\(F\)if none pass\. GRPO uses a soft rewardlog⁡σ⁡\(sF\)\+mass\_term\\log\\sigma\(s\_\{F\}\)\+\\texttt\{mass\\\_term\}versussF\+mass\_terms\_\{F\}\+\\texttt\{mass\\\_term\}, and unparsed reward−100\-100\. The TRL 1\.12\.0 GRPOTrainer recipe uses its default DAPO token\-normalised loss and group reward scaling, not the original sequence\- normalised GRPO loss\. Group size88, LoRAr=16r=16, two epochs over10001000prompts, lr10−510^\{\-5\},β=0\.04\\beta=0\.04, temperature0\.90\.9, top\-p1\.01\.0for training, completion limit700700, per\-device completion batch88, accumulation22, yield10001000optimizer steps; evaluation uses top\-p0\.950\.95\.

## Appendix H7B and online GRPO experiments: registration and support

Registered 2026\-09\-15 before the runs\. P\-A1 held: hard\-gated DPO at 7B improves per\-draw and two\-stage composed answer on each seed, ratio1\.79×1\.79\\timesinside\[1\.3,2\.3\]\[1\.3,2\.3\]\. P\-A2 held: feasibility is lower on every gated seed\. P\-A3 is partial: exact identity,Across∈\[0\.45,0\.58\]A\_\{\\rm cross\}\\in\[0\.45,0\.58\]and sum\-armq≥0\.9q\\geq 0\.9all hold;Aown≥0\.93A\_\{\\rm own\}\\geq 0\.93on every arm fails on gated seed 1 \(0\.91930\.9193parsed\-only,0\.92500\.9250including unparsed failures\)\. No run is removed or threshold changed\.

Table 6:Parsed\-only own\-bin support for the added 7B runs\. Bootstrap intervals are post\-hoc percentile intervals from 3000 requirement\-level resamples, seed 20260918, not training\-seed intervals; pairs within a requirement are dependent\.An empirical interval of\[1,1\]\[1,1\]with only 11 supported requirements indicates no observed errors, not proof of perfect ordering\. The interval for the failed seed includes0\.930\.93but does not change the registered outcome\. The intersection of requirements supportingAownA\_\{\\rm own\}in all six matched 7B arms contains only one requirement, so restricting every arm to a common support would replace one support problem with a more severe one\. Identity statistics therefore average each arm over its informative pools; the direct feasibility audit in Appendix[M](https://arxiv.org/html/2610.29029#A13)uses4040–225225informative requirements per seed and gives AUC0\.9400\.940–0\.9910\.991\.

GRPO predictions: feasibility above0\.80\.8\(P\-B1\) and sum per\-draw in\[0\.08,0\.18\]\[0\.08,0\.18\]held; supply advantage at least0\.100\.10\(P\-B2\) failed\. Gate per\-draw above the sum on every seed held, but gate per\-draw at least0\.200\.20\(P\-B3\) failed\. Coordinate sd below0\.170\.17\(P\-B4\) held for both rewards\. All parse rates exceed0\.9980\.998, so no engineering discard applies\. The strict no\-difference branch is not supported for supply, but this is a modest online result, not a large two\-algorithm replication\. Endpoint outcomes do not establish the predicted reward\-optimisation trajectory\.

Table 7:Training scope checks: a second DPO round and online GRPO\. “Composed” uses the same two\-stage evaluator as Table[39](https://arxiv.org/html/2610.29029#A32.T39); mean±\\pmsd over three seeds\.
## Appendix IRegistered predictions and outcomes

Table 8:Evidence chronology for the follow\-up studies\. “Pre\-specified” means the named plan/protocol was committed before the corresponding new analysis or training run; it does not imply that earlier results in the paper were unknown\.Committed predictions are reported with misses rather than filtered post hoc\. The chronology above distinguishes original results, prospective follow\-ups, and later post\-hoc or non\-preregistered checks\. The detailed registered statements follow\. \(1\) Identity within0\.030\.03withAcross∈\[0\.45,0\.58\]A\_\{\\rm cross\}\\in\[0\.45,0\.58\]on every engine arm: partially held \(mean0\.0160\.016, max0\.0570\.057, so not every arm is within0\.030\.03; cross0\.470\.47–0\.560\.56on the original arms\)\. \(2\) Variance\-normalised sums belowη=0\.5\\eta=0\.5, gates above0\.850\.85: held for the z\-scorings and the gates; exceeded by0\.030\.03by the rank average; a fixed weight of0\.0050\.005, identified in a later unregistered ablation, was not registered and matches the gates\. \(3\) Argmax dominance of the logit separating trained from base pools: failed \(dominance is0\.700\.70–0\.930\.93on every arm\)\. \(4\) Round\-1 gated pairs: per\-draw above0\.0950\.095, supply above0\.4600\.460, feasible below0\.880\.88: held\. Registered before the matched control existed; against the matched control the gated advantage is2\.3×2\.3\\times, not3\.3×3\.3\\times\. \(5\) Round\-2 sum pairs: per\-draw at or below0\.100\.10: failed \(0\.1550\.155\); supply fell instead\. \(6\) GSM8K: boxed rate above0\.850\.85andqqabove0\.850\.85: held \(0\.980\.98,0\.960\.96\)\.Across∈\[0\.50,0\.65\]A\_\{\\rm cross\}\\in\[0\.50,0\.65\]: missed low by0\.020\.02on some seeds \(0\.470\.47–0\.510\.51\)\. Composite: the registered no\-cliff branch \(spread ratio below0\.50\.5, rules agree within0\.10\.1\) obtained\. Gate pairs carry more correctness contrast than sum pairs: failed \(identical pairs\)\. The fall in best\-of\-1616was not registered\. \(7\) Strong training controls \(T1–T5, Appendix[O](https://arxiv.org/html/2610.29029#A15)\): tuned sum and log\-prob within0\.040\.04of the gate — failed, both below by0\.190\.19and0\.080\.08; mass\-only below its falsifier — held; every new arm above the unit sum — failed;q\>0\.9q\>0\.9on tuned\-sum arms — failed \(0\.570\.57–0\.630\.63\),Aown\>0\.9A\_\{\\rm own\}\>0\.9— held\. \(8\) Pre\-specified later\-arm forecast on the twelve new arms \(F1–F6, Appendix[S](https://arxiv.org/html/2610.29029#A19)\): error≤0\.05\\leq 0\.05— held \(0\.0200\.020\); persistence\>0\.10\>0\.10on the non\-saturated subset — missed by0\.0070\.007; AUC monotone inqqwith mass\-only highest — held on the mean, missed on one seed \(0\.7370\.737against a registered\>0\.80\>0\.80\);η≥0\.5\\eta\\geq 0\.5everywhere — held, but the gate ties the tuned sum under the raw selector; three\-conjunct terms unchanged — held\. \(9\) Pair\-contrast sweep and conditioning of the new arms \(C1–C4, Appendices[O](https://arxiv.org/html/2610.29029#A15),[W](https://arxiv.org/html/2610.29029#A23),[X](https://arxiv.org/html/2610.29029#A24)\): no weight within0\.850\.85of the gate’s contrast — failed \(0\.910\.91atw≈0\.1w\\approx 0\.1\); mass gain0\.950\.95for tuned sum and mass\-only,0\.450\.45for log\-prob — all failed at0\.720\.72–0\.740\.74; printability below0\.800\.80under mass\-only — held \(0\.570\.57\); envelope conditioning by rule — held on all four; no second\-level collapse — held; mass\-trackers hottest — held; every arm above base survival — failed on two seeds\. \(10\) Three\-conjunct DPO \(Appendix[X](https://arxiv.org/html/2610.29029#A24)\): seven of eleven held, falsifier not fired, all misses toward a smaller effect\. \(11\) Contrast\-matched sum, 1\.5B \(Appendix[O](https://arxiv.org/html/2610.29029#A15)\): decision rule returned “contrast suffices”; every registered point value \(feasible0\.790\.79, per draw0\.270\.27, supply0\.760\.76, composed0\.700\.70, gain0\.800\.80,qq0\.900\.90, envelopeρ<0\.45\\rho<0\.45\) missed upward — the policy reached the gate;Aown\>0\.9A\_\{\\rm own\}\>0\.9and the base\-pool forecast within0\.040\.04held\. A later, non\-preregistered 7B follow\-up atw=0\.05w=0\.05is reported in Appendix[O](https://arxiv.org/html/2610.29029#A15); it is not counted as a registered outcome\. \(12\) Within\-run forecast \(Appendix[S](https://arxiv.org/html/2610.29029#A19)\): next\-checkpoint error≤0\.06\\leq 0\.06with persistence worse and≥14/18\\geq 14/18wins — held \(0\.0490\.049,0\.0960\.096,15/1815/18;0\.0210\.021against0\.0640\.064with pooled source terms, an unregistered variant\);q≥0\.8q\\geq 0\.8by step500500on the saturating runs and persistence worst on the first transition — held;Aown≥0\.9A\_\{\\rm own\}\\geq 0\.9at every checkpoint — failed once \(0\.8530\.853\); tuned\-sumq\>0\.5q\>0\.5by step500500— missed by0\.0020\.002; base\-pool error≤0\.07\\leq 0\.07— held \(0\.0320\.032\); null arm — held\.

## Appendix JClosest\-work map

Table 9:Closest literatures and the distinction relevant to this paper\. We separate ingredients that are classical from the new verifier\-audit consequence\.
## Appendix KCompute

Original GPU experiments ran on single A100\-80GB nodes; the added 7B jobs allowed A100\-80GB or H100\. Actual accelerator assignment for the added 7B jobs is not recoverable from the packaged logs, so we do not attribute those runs to a more specific GPU model\. Original DPO at 1\.5B:≈2\.5\\approx 2\.5h per seed; at 7B:≈5\\approx 5h\. Generating1 0001\\,000engine requirements atK=16K\{=\}16:≈4\.5\\approx 4\.5h at 1\.5B; GSM8K500×64500\\times 64:≈3\\approx 3h\. The cVAE arms, latent search, oracle scoring and every analysis of §[4](https://arxiv.org/html/2610.29029#S4)–[AF](https://arxiv.org/html/2610.29029#A32)ran on CPU; re\-labelling a16 00016\\,000\-candidate pool takes2020s on2020cores\.

## Appendix LAdditional geometry observations

#### What the original 1\.5B designers propose\.

The on\-target criterion can be met by a degenerate body — the constrained autoencoder of §[7](https://arxiv.org/html/2610.29029#S7)met it with a block — so we grade the policies’ on\-target designs in parameter space against the prior’s \(Appendix[E](https://arxiv.org/html/2610.29029#A5), Figure[5](https://arxiv.org/html/2610.29029#A12.F5)\)\. None are blocks: across6 4006\\,400–17 00017\\,000on\-target designs per policy, no design has every radius control near its bound,00–1%1\\%of radius controls sit at a bound \(prior3%3\\%\), the mean radius control is0\.420\.42–0\.490\.49\(prior0\.500\.50\), every design is distinct at two decimals, bodies use2\.92\.9zones on average, and the oracle’s printability score is1\.01\.0\. What training changes is diversity: the per\-coordinate spread of on\-target designs falls from0\.280\.28\(prior\) and0\.230\.23\(base\) to0\.170\.17after round 1 under either score, and to0\.070\.07after a sum\-ranked second round, whose designs use two of the four lattice families; the gate\-ranked second round keeps0\.130\.13and three\. The narrowing of Figure[3](https://arxiv.org/html/2610.29029#A4.F3)is visible in the geometry\.

Figure 5:One on\-target design from each policy on the same requirement, decoded by the function the solver grades: longitudinal section \(bore and wall\) and mid\-length cross\-section through the lattice\. The policies propose engines with different lattice families and wall profiles, not blocks\.

## Appendix MPost\-hoc selection and uncertainty audit

Table 10:Selection efficiencyη\\etaatK=16K=16on identical engine pools using the same critic\. The raw unit sum performs poorly after feasibility saturates\. Rules that make the feasibility contribution flat in the saturated region recover most of the available selection lift\. Mean±\\pmsd over seeds where applicable\.Table 11:AUC\-preserving scale sensitivity on the matched 1\.5B sum\-DPO pools\. Multiplying the feasibility logit by positiveccpreserves its marginal ROC/AUC exactly, yet changes the additive selector’s best\-of\-16 answer\. Candidate pools and critic outputs are fixed\.This follow\-up uses stored test pools; it is post\-hoc, not a new registered prediction\. We compute AUC directly against oracle feasibility on parsed samples, using every feasible sample as positive, including feasible mass misses\. Across the matched DPO arms it is0\.9400\.940–0\.9910\.991, with4040–225225informative requirements per seed\. Precision of the predicted passing set is0\.9610\.961–0\.9810\.981\. These measurements supplement own\-bin AUC, which only uses on\-target positives\. A fixed held\-out score alone does not establish on\-policy reliability\.

Table 12:Right\-minus\-sum differences on the same test requirements\. DPO right arm: hard\-gated ranking; GRPO right arm: log\-probability reward\. Intervals use 2000 paired requirement bootstrap resamples after averaging the three fixed training seeds within each requirement\. They quantify test\-requirement variation conditional on those seeds, not population uncertainty over training seeds\. Seed SD is reported separately in the training tables\.#### Residual spread is not uniformly noise\.

We permute feasibility logits within each requirement’s predicted passing set, preserving the set and mass scores, and re\-evaluate the raw sum \(100 permutations\)\. On gated 7B pools mean raw answer falls from0\.2370\.237to0\.2090\.209: residual ordering retains useful signal\. This does not make the raw sum competitive with two\-stage selection, but it precludes treating all residual feasibility variation as irrelevant noise\. Permutation is a conditional score ablation, not an intervention on training\.

#### Parsing and the second\-round lineage\.

Relative to the single parent policy, sum\-ranked children lose supply even on requirements where both parent and child pools are fully parsed: differences−0\.141\-0\.141,−0\.110\-0\.110,−0\.061\-0\.061on915915,762762,685685requirements\. Gated children improve supply on the corresponding subsets by0\.0500\.050,0\.1030\.103,0\.2180\.218\. This shows that format breakage alone cannot account for all observed loss; the subsets are post\-hoc and selected, so this is not a causal correction for parsing\. Within\-requirement coordinate spread is reported in the main text conditional on at least two successful samples\.

#### Top\-1 failures and sampling budget\.

The analysis artifact includes nestedK=1,2,4,8,16K=1,2,4,8,16answer curves, failures split into unparsed, infeasible parsed, and feasible mass misses, and the fraction of requirements with a solution that the selector misses\. These directly measure the top of the ranking, which AUC alone does not determine\. For matched sum DPO at 1\.5B, the raw sum misses an available solution on0\.4460\.446–0\.4820\.482of requirements, compared with0\.0350\.035–0\.0480\.048under two\-stage selection\.

## Appendix NComplete original mixture table

Table 13:Eq\.[3](https://arxiv.org/html/2610.29029#S4.E3)on the head that scores the learned conjunct, the original policy\-sampled pool, parsed candidates only; the identity is evaluated per requirement before aggregation, and rows with±\\pmare mean±\\pmsd over three training seeds\. “Predicted” is\(1−q\)​Aown\+12​q\(1\-q\)A\_\{\\rm own\}\+\\tfrac\{1\}\{2\}q, i\.e\. the identity withAcrossA\_\{\\rm cross\}set to12\\tfrac\{1\}\{2\}; the one constant it uses,AownA\_\{\\rm own\}, is measured on the same pool\. The engine’s 7B base has parse rate0\.760\.76, the 1\.5B base0\.970\.97; all other engine rows≥0\.95\\geq 0\.95\.
## Appendix OMatched train\-pair follow\-up: tuned sum, log\-probability and mass\-only DPO

#### Protocol, fixed before training\.

The protocol, committed asPROTOCOL\_STRONG\_CONTROLS\_2026\-09\-18\.md, was written after the original sum\-versus\-gate result was known and before any new pool or policy existed; it is a prospective follow\-up, not an original preregistration\. A new Qwen2\.5\-1\.5B\-Instruct pool on all500500committed validation requirements \(K=16K\{=\}16, seed 0, the same shots and critic\) selects a weight from the thirteen\-point grid00,0\.00050\.0005,0\.0010\.001,0\.0020\.002,0\.0050\.005,0\.010\.01,0\.020\.02,0\.050\.05,0\.10\.1,0\.20\.2,0\.50\.5,11,22by the highest oracle\-labelled validation top\-1 answer; no test label enters\. The sweep peaks atw=0\.005w=0\.005\(0\.2380\.238\) and is monotone on either side \(0\.1660\.166atw=0w=0,0\.2080\.208at0\.010\.01,0\.1340\.134at0\.10\.1,0\.0900\.090at11\)\. Three rules — the weighted sum at that weight,log⁡σ⁡\(logit\)\+mass\_term\\log\\sigma\(\\texttt\{logit\}\)\+\\texttt\{mass\\\_term\}, andmass\_termalone — were then trained on the stored1000×161000\\times 16train pool with exactly the original recipe \(4 0004\\,000pairs, four per prompt,β=0\.1\\beta=0\.1, three epochs, lr5×10−55\\times 10^\{\-5\}, LoRAr=16r=16\), three seeds each, no seed dropped, and evaluated on the committed1 0001\\,000test requirements with common selectors\. All nine pools were re\-labelled by the oracle independently of the bench\.

#### Pair content\.

We reconstruct the original sum pairs exactly from the stored train pool and build the other rules’ pairs with the same top\-four versus bottom\-four builder\. Table[14](https://arxiv.org/html/2610.29029#A15.T14)gives the oracle content of each rule’s pairs and Table[15](https://arxiv.org/html/2610.29029#A15.T15)the policies trained on them\.

Table 14:Matched train\-pair content, chosen/rejected: feasible rate, on\-target rate and median absolute oracle log\-mass error \(parsed sides\)\. Exact triple overlap \(requirement, chosen, rejected\) is0\.0990\.099between sum and gate,0\.5860\.586between sum and log\-prob,0\.2100\.210between gate andw=0\.005w=0\.005, and0\.4630\.463betweenw=0\.005w=0\.005and mass\-only\.Table 15:Policies trained on the five rules; test1000×161000\\times 16, mean±\\pmseed sd\. The last three columns are the gate’s advantage under the common two\-stage selector with a paired requirement bootstrap \(seeds fixed\), and the head’s mixture terms on the trained policy’s pool\.
#### Registered predictions and outcomes\.

Before any result was read \(PREREG\_TUNED\_SUM\_THEORY\_2026\-09\-18\.md\) we registered, from the aggregation argument of §[AE](https://arxiv.org/html/2610.29029#A31), that the tuned sum and the log\-probability form would each land within0\.040\.04of the gate’s composed answer and0\.050\.05of its per\-draw rate and supply \(T1, T2\), that mass\-only would stay below0\.700\.70composed with the falsifier “within0\.030\.03of the gate” \(T3\), that every new arm would beat the unit sum \(T4\), and thatq\>0\.9q\>0\.9andAown\>0\.9A\_\{\\rm own\}\>0\.9on every tuned\-sum and log\-prob arm \(T5\)\. T3 held on every seed and the falsifier did not fire\. T1, T2 and T4 missed: the tuned sum is0\.1870\.187below the gate, the log\-probability form0\.0820\.082below, and the tuned sum and mass\-only fall below the unit sum on composed answer\. T5 missed onqqfor the tuned sum \(0\.570\.57–0\.630\.63\) withAown\>0\.92A\_\{\\rm own\}\>0\.92everywhere\. The prediction that failed was the extrapolation of a selection\-side argument to pair construction; the selection\-side part held \(the validation\-selected weight lies in the registered band\[0\.001,0\.005\]\[0\.001,0\.005\]\)\.

#### Why the tuned weight is the wrong training weight\.

Figure[8](https://arxiv.org/html/2610.29029#A17.F8)sweepswwover the train pool\. Among parsed train candidates the logit’s spread \(9\.969\.96\) is ten times the mass term’s \(0\.970\.97\), so the pair contrast in feasibility,Δ​F\\Delta F, falls monotonically aswwshrinks while the mass contrastΔ​M\\Delta Mrises:\(0\.64,0\.11\)\(0\.64,0\.11\)atw=1w=1,\(0\.13,0\.82\)\(0\.13,0\.82\)at0\.0050\.005,\(0\.03,0\.84\)\(0\.03,0\.84\)at00\. The gate sits at\(0\.48,0\.63\)\(0\.48,0\.63\)\. The scalar curve passes close to that point nearw≈0\.1w\\approx 0\.1\(\(0\.48,0\.52\)\(0\.48,0\.52\)\), a weight20×20\\timesthe validation\-selected one and one the selection criterion ranks fifth from the bottom of thirteen; we had registered that no weight would come within0\.850\.85of the gate on both contrasts, and the sweep refutes that \(0\.910\.91atw=0\.075w=0\.075\)\. The chosen sets ofw=0\.1w=0\.1and of the gate overlap65%65\\%, the full pairs15%15\\%: the gate’s rejected candidates are the infeasible ones with the worst mass term, while a scalar score rejects by whichever term dominates at that weight\.

#### The contrast\-matched weight matches the observed gate endpoint\.

Whether that weight recovers the gate’s outcome was registered before training \(PLAN\_CONTRAST\_DPO\_2026\-09\-19\.md:w=0\.1w=0\.1at 1\.5B, three seeds, recipe unchanged\), with a decision rule: mass gain≥0\.95\\geq 0\.95and composed≥0\.74\\geq 0\.74would mean contrast suffices and the gate is a weight in disguise; gain≤0\.78\\leq 0\.78and composed≤0\.66\\leq 0\.66at feasibility≥0\.75\\geq 0\.75would mean the rejected set matters\. The rule returned*contrast suffices*: per draw0\.312±0\.0210\.312\\pm 0\.021, feasible0\.8670\.867, supply0\.821±0\.0170\.821\\pm 0\.017, composed0\.770±0\.0230\.770\\pm 0\.023, mass gain0\.98±0\.020\.98\\pm 0\.02,qq0\.9610\.961,AownA\_\{\\rm own\}0\.9270\.927\(Table[15](https://arxiv.org/html/2610.29029#A15.T15)\); the paired gate\-minus\-sum differences are−0\.001\-0\.001\[−0\.015,0\.012\]\[\-0\.015,0\.012\]per draw,−0\.014\-0\.014\[−0\.031,0\.003\]\[\-0\.031,0\.003\]supply,−0\.012\-0\.012\[−0\.030,0\.006\]\[\-0\.030,0\.006\]composed and−0\.020\-0\.020\[−0\.039,−0\.001\]\[\-0\.039,\-0\.001\]under the log\-probability selector\. Every registered point value missed, all in the same direction: we had expected a partial recovery \(feasible0\.790\.79, per draw0\.270\.27, composed0\.700\.70, gain0\.800\.80, envelopeρ<0\.45\\rho<0\.45\) and the policy reached the gate on every count and installed envelope conditioning as well \(0\.69±0\.060\.69\\pm 0\.06\), which no other rule does together with unit budget tracking\. The0\.720\.72plateau of the three mis\-weighted scalar rules is therefore a property of their contrast, not of scalar scores, and the gate’s distinctive rejected set does not matter\. The base\-pool forecast of §[4](https://arxiv.org/html/2610.29029#S4)lands within0\.0240\.024on these arms\. The 7B base train pool is used below to test whether the weight\-selection criterion itself transfers; a later follow\-up then trains the corresponding 7B scalar arm\.

#### Choosing the contrast weight without gold labels\.

The trainedw=0\.1w=0\.1arm above was selected by oracle\-unit contrastsΔ​F\\Delta FandΔ​M\\Delta M\. We subsequently pre\-specified a label\-free repetition using only the critic on the same train pool:Δ​P\\Delta Pis the difference in predicted feasibility probability between chosen and rejected samples, andΔ​M^\\Delta\\widehat\{M\}is the reduction in predicted absolute log\-mass error\. Across the weight grid,Δ​P\\Delta PandΔ​F\\Delta Fhave Spearman correlation1\.0001\.000at 1\.5B and0\.9990\.999at 7B;Δ​M^\\Delta\\widehat\{M\}andΔ​M\\Delta Mhave0\.9950\.995and0\.9990\.999\. Matching the hard gate in critic units selectsw=0\.075w=0\.075at 1\.5B andw=0\.056w=0\.056–0\.0750\.075at 7B, the same narrow region as the oracle\-unit sweep\.

A further post\-hoc criterion removes even the gate as a reference\. For eachww, we normaliseΔ​P\\Delta PandΔ​M^\\Delta\\widehat\{M\}by their respective maxima over the sweep and maximise the smaller of the two normalised contrasts\. This intrinsic maximin rule selectsw=0\.075w=0\.075at 1\.5B andw=0\.056w=0\.056at 7B; maximising their product or minimising distance to the ideal point\(1,1\)\(1,1\)selects exactly the same weights\. This criterion by itself does not prove that the resulting policy must match a gate; the 1\.5B experiment above and the later 7B follow\-up below test that training claim\. It answers the practical weight\-selection question using critic outputs alone and is distinct from the failed label\-free*diagnostic*in Appendix[T](https://arxiv.org/html/2610.29029#A20)\.

#### Post\-hoc split stability of the intrinsic criterion\.

As a sampling\-sensitivity check, we recompute the critic\-only maximin rule on the independently generated*base test*pools, which were not used to build the training pairs\. It selects exactly the train\-pool choices:w=0\.074989w=0\.074989at 1\.5B andw=0\.056234w=0\.056234at 7B\. In 500 requirement\-bootstrap resamples of each test pool, the 1\.5B maximin choice is0\.0749890\.074989in499/500499/500resamples \(and0\.10\.1once\); at 7B it is0\.0562340\.056234in415/500415/500,0\.050\.05in66/50066/500and0\.0749890\.074989in19/50019/500\. Thus every resample at both scales stays in the same\[0\.05,0\.10\]\[0\.05,0\.10\]region\. Product and distance\-to\-ideal criteria choose the same full\-test\-pool points\. This audit is post\-hoc and is evidence for split stability of the criterion, not a new policy training result\.

#### 7B follow\-up of contrast\-matched training\.

We subsequently trained three 7B DPO seeds atw=0\.05w=0\.05, the nearest implemented weight to the intrinsic 7B choice near0\.0560\.056, with the same4 0004\\,000\-pair recipe\. This run occurred after the 1\.5B contrast result and is not preregistered\. The three policies reach target success per draw0\.218±0\.0210\.218\\pm 0\.021and supply@160\.615±0\.0680\.615\\pm 0\.068\. Re\-evaluating the stored pools with the common two\-stage selector gives composed answer0\.565±0\.0650\.565\\pm 0\.065, compared with0\.562±0\.0260\.562\\pm 0\.026for the matched 7B hard gate\. The paired requirement bootstrap for contrast\-matched minus gate is0\.0030\.003\[−0\.015,0\.020\]\[\-0\.015,0\.020\]; against the 7B unit\-sum policy it is0\.1510\.151\[0\.126,0\.175\]\[0\.126,0\.175\]\. The scalar’s own weighted\-sum selector reaches only0\.261±0\.0450\.261\\pm 0\.045, so the result supports transfer of the pair\-construction criterion without collapsing the distinction between training and inference\-time selection\. The follow\-up export retained target labels, critic scores, parse masks, and design vectors, but not the separate oracle\-feasibility label; we therefore do not report a follow\-up feasibility rate in Table[39](https://arxiv.org/html/2610.29029#A32.T39)\.

Figure 6:Label\-free pair contrast in critic units over scalar weights\. The balanced region is nearw≈0\.05w\\approx 0\.05–0\.10\.1at both scales, far from the validation\-tunedw=0\.005w=0\.005\. Oracle labels are not used by these axes\.

## Appendix PProof of the sharp AUC identified set

LetS\+S^\{\+\}denote a score drawn from the positive class andSj−S\_\{j\}^\{\-\}a score drawn from negative failure binBjB\_\{j\}\. With the usual half\-credit convention for ties,aj=Pr\[S\+\>Sj−\]\+12Pr\[S\+=Sj−\]a\_\{j\}=\\Pr\[S^\{\+\}\>S\_\{j\}^\{\-\}\]\+\\tfrac\{1\}\{2\}\\Pr\[S^\{\+\}=S\_\{j\}^\{\-\}\]\. Drawing a negative first chooses binjjwith probabilityπj\\pi\_\{j\}, so the law of total probability givesA=∑jπj​aj=π⊤​aA=\\sum\_\{j\}\\pi\_\{j\}a\_\{j\}=\\pi^\{\\top\}a\. Hence every data\-generating process compatible with\(A,π\)\(A,\\pi\)lies in\{a∈\[0,1\]J:π⊤​a=A\}\\\{a\\in\[0,1\]^\{J\}:\\pi^\{\\top\}a=A\\\}\.

The set is sharp\. For anya∈\[0,1\]Ja\\in\[0,1\]^\{J\}satisfying the equality, takeS\+∼Unif⁡\(0,1\)S^\{\+\}\\sim\\mathrm\{Unif\}\(0,1\)\. Foraj∈\(0,1\)a\_\{j\}\\in\(0,1\)choose the negative\-bin score to be the constantSj−=1−ajS\_\{j\}^\{\-\}=1\-a\_\{j\}; for the endpoints choose any constant above11or below00\. Then the positive\-versus\-bin AUC is exactlyaja\_\{j\}for every bin simultaneously, so every point in the set is attainable\. ForJ=2J=2,A=\(1−q\)​Aown\+q​AcrossA=\(1\-q\)A\_\{\\rm own\}\+qA\_\{\\rm cross\}and eliminatingAcross∈\[0,1\]A\_\{\\rm cross\}\\in\[0,1\]yields Eq\.[4](https://arxiv.org/html/2610.29029#S4.E4); exchanging the roles gives the symmetric bound\. Ifq=1q=1,AownA\_\{\\rm own\}drops out completely\. This proves Proposition[1](https://arxiv.org/html/2610.29029#Thmproposition1)\.

Figure 7:Complete erasure for the two\-bin audit\. In the shaded regionq≥max⁡\(A,1−A\)q\\geq\\max\(A,1\-A\), the sharp identified interval forAownA\_\{\\rm own\}is all of\[0,1\]\[0,1\]\.#### Controlled\-mixture audit design\.

Suppose the bin\-specific ordering vectoraais held fixed while an auditor constructsKKnegative mixtures with known rowsπ\(k\)\\pi^\{\(k\)\}\. WritingΠ\\Pifor the resultingK×JK\\times Jmatrix, the aggregate audits satisfyA=Π​aA=\\Pi a\. Henceaais point\-identified for every possible value iffrank⁡\(Π\)=J\\operatorname\{rank\}\(\\Pi\)=J; otherwise a null\-space direction that remains inside\[0,1\]J\[0,1\]^\{J\}is observationally equivalent\. For the binary split, two audits withq1≠q2q\_\{1\}\\neq q\_\{2\}give

Aown=q2​A1−q1​A2q2−q1,Across=\(1−q1\)​A2−\(1−q2\)​A1q2−q1\.A\_\{\\rm own\}=\\frac\{q\_\{2\}A\_\{1\}\-q\_\{1\}A\_\{2\}\}\{q\_\{2\}\-q\_\{1\}\},\\qquad A\_\{\\rm cross\}=\\frac\{\(1\-q\_\{1\}\)A\_\{2\}\-\(1\-q\_\{2\}\)A\_\{1\}\}\{q\_\{2\}\-q\_\{1\}\}\.This is a controlled\-reweighting result; it does not license combining ordinary policy checkpoints whose conditional score distributions may themselves change\.

A useful corollary is the exact condition for complete loss of information aboutAownA\_\{\\rm own\}whenq<1q<1: its sharp interval is\[0,1\]\[0,1\]iff1−q≤A≤q1\-q\\leq A\\leq q\. All four matched engine rows in Table[2](https://arxiv.org/html/2610.29029#S4.T2)satisfy this condition\. The result is an identification statement about the aggregate statistic; it does not say that direct own\-bin estimates are unreliable when their support is adequate\.

## Appendix QWhy validation\-tuned selection weights need not be pair\-optimal

For two candidatesi,ji,jscored byrw=w​sF\+sMr\_\{w\}=ws\_\{F\}\+s\_\{M\}, their order can change only when

w⁡\(sF,i−sF,j\)\+\(sM,i−sM,j\)=0\.w\(s\_\{F,i\}\-s\_\{F,j\}\)\+\(s\_\{M,i\}\-s\_\{M,j\}\)=0\.Thus, away from the finite set of pairwise crossing weights, every within\-pool ranking is fixed\. Any top\-kkselector and any deterministic top\-versus\-bottom pair builder is consequently piecewise constant inww\. If two rules induce the same chosen–rejected multiset, the DPO loss is literally the same function of the policy parameters because the scalar score is no longer an input to DPO\. This proves Proposition[7](https://arxiv.org/html/2610.29029#Thmproposition7)\.

For the approximate statement, write the per\-pair DPO loss asℓθ​\(z\)=log⁡\(1\+exp⁡\[−β​dθ​\(z\)\]\)\\ell\_\{\\theta\}\(z\)=\\log\(1\+\\exp\[\-\\beta d\_\{\\theta\}\(z\)\]\)\. If\|dθ​\(z\)\|≤R\|d\_\{\\theta\}\(z\)\|\\leq R, thenℓθ\\ell\_\{\\theta\}ranges over an interval of width exactlyβ​R\\beta R\. The variational characterisation of total variation therefore gives\|𝔼P​ℓ−𝔼Q​ℓ\|≤β​R​dTV​\(P,Q\)\|\\mathbb\{E\}\_\{P\}\\ell\-\\mathbb\{E\}\_\{Q\}\\ell\|\\leq\\beta Rd\_\{\\rm TV\}\(P,Q\)\. For the gradient bound, project the gradient difference onto an arbitrary unit vector; the projected per\-pair gradient lies in\[−G,G\]\[\-G,G\], so its expectation changes by at most2​G​dTV​\(P,Q\)2Gd\_\{\\rm TV\}\(P,Q\)\. Taking the supremum over unit vectors proves Proposition[7](https://arxiv.org/html/2610.29029#Thmproposition7)\. For uniform finite pair multisets, total variation equals one minus their overlap coefficient \(counting multiplicity\)\.

#### A local bridge from pair contrasts to the first DPO update\.

###### Proposition 4\(Reference\-gradient contrast bound\)\.

Letθ0\\theta\_\{0\}equal the DPO reference policy, letg⁡\(z\)=∇θdθ​\(z\)\|θ0g\(z\)=\\nabla\_\{\\theta\}d\_\{\\theta\}\(z\)\|\_\{\\theta\_\{0\}\}, and letϕ⁡\(z\)∈ℝd\\phi\(z\)\\in\\mathbb\{R\}^\{d\}be a vector of pair\-property contrasts\. Supposeg⁡\(z\)=B​ϕ​\(z\)\+r⁡\(z\)g\(z\)=B\\phi\(z\)\+r\(z\), with𝔼P​‖r‖≤ϵP\\mathbb\{E\}\_\{P\}\\\|r\\\|\\leq\\epsilon\_\{P\}and𝔼Q​‖r‖≤ϵQ\\mathbb\{E\}\_\{Q\}\\\|r\\\|\\leq\\epsilon\_\{Q\}\. Then

‖∇LP​\(θ0\)−∇LQ​\(θ0\)‖≤β2​\(‖B‖​‖𝔼P​ϕ−𝔼Q​ϕ‖\+ϵP\+ϵQ\)\.\\\|\\nabla L\_\{P\}\(\\theta\_\{0\}\)\-\\nabla L\_\{Q\}\(\\theta\_\{0\}\)\\\|\\leq\\frac\{\\beta\}\{2\}\\Bigl\(\\\|B\\\|\\,\\\|\\mathbb\{E\}\_\{P\}\\phi\-\\mathbb\{E\}\_\{Q\}\\phi\\\|\+\\epsilon\_\{P\}\+\\epsilon\_\{Q\}\\Bigr\)\.

For DPO,ℓθ​\(z\)=log⁡\(1\+exp⁡\[−β​dθ​\(z\)\]\)\\ell\_\{\\theta\}\(z\)=\\log\(1\+\\exp\[\-\\beta d\_\{\\theta\}\(z\)\]\)\. At the reference policyθ0\\theta\_\{0\},dθ0​\(z\)=0d\_\{\\theta\_\{0\}\}\(z\)=0for every pair, hence

∇LP​\(θ0\)=−β2​𝔼P​\[g⁡\(z\)\]\.\\nabla L\_\{P\}\(\\theta\_\{0\}\)=\-\\frac\{\\beta\}\{2\}\\,\\mathbb\{E\}\_\{P\}\[g\(z\)\]\.Subtracting the expressions forPPandQQ, applying the triangle inequality and then Jensen proves Proposition[4](https://arxiv.org/html/2610.29029#Thmproposition4)\. The assumption is an explicit local feature model for the preference\-log\-ratio gradient\. We did not retain the per\-pair gradient artifacts needed to estimate its residual for these runs, so the proposition is a sufficient\-condition bridge, not an empirically established mechanism or a claim that oracle contrasts are sufficient statistics for later optimization\.

There is nevertheless no implication from validation top\-1 optimality to balanced training\-pair contrast\. A two\-prompt counterexample suffices\. Each row below is a candidate with gold conjuncts\(F,M\)\(F,M\)and proxy components\(sF,sM\)\(s\_\{F\},s\_\{M\}\):

123prompt A\(0,1,5,1\)\(1,1,0,6\)\(0,1,2,0\)prompt B\(0,1,5,2\)\(0,0,3,4\)\(1,1,4,5\)\.\\begin\{array\}\[\]\{c\|ccc\}&1&2&3\\\\ \\hline\\cr\\text\{prompt A\}&\(0,1;5,1\)&\(1,1;0,6\)&\(0,1;2,0\)\\\\ \\text\{prompt B\}&\(0,1;5,2\)&\(0,0;3,4\)&\(1,1;4,5\)\.\\end\{array\}Atw=0\.5w=0\.5, the successful\(1,1\)\(1,1\)candidate is top\-ranked for both prompts, so top\-1 success is11, but top\-versus\-bottom pairs have average oracle contrasts\(Δ​F,Δ​M\)=\(1,0\)\(\\Delta F,\\Delta M\)=\(1,0\)\. Atw=2w=2, top\-1 success falls to1/21/2, while the average contrasts become\(1/2,1/2\)\(1/2,1/2\)\. Hence a weight that is strictly better for selection can erase one training contrast that a worse selector preserves\. The example is only an existence result; the engine experiment tests whether this mismatch matters in the observed system\.

#### Pair overlap is not the empirical explanation\.

All 1\.5B round\-1 rules use4 0004\\,000pairs from the same train pool\. Their full\-pair overlap with the hard gate and eventual composed answer are

ranking rulepair overlap with gatecomposed @16unit sum0\.0990\.596validation\-tuned sum0\.2100\.571contrast\-matched sum​\(w=0\.1\)0\.1500\.770log\-probability0\.1400\.675mass\-only0\.2620\.519\\begin\{array\}\[\]\{lcc\}\\hline\\cr\\hline\\cr\\text\{ranking rule\}&\\text\{pair overlap with gate\}&\\text\{composed @16\}\\\\ \\hline\\cr\\text\{unit sum\}&0\.099&0\.596\\\\ \\text\{validation\-tuned sum\}&0\.210&0\.571\\\\ \\text\{contrast\-matched sum \}\(w=0\.1\)&0\.150&0\.770\\\\ \\text\{log\-probability\}&0\.140&0\.675\\\\ \\text\{mass\-only\}&0\.262&0\.519\\\\ \\hline\\cr\\hline\\cr\\end\{array\}Thus the best\-performing scalar is not the one whose examples most resemble the gate’s\. Its chosen\-candidate set overlaps the gate by65%65\\%, but only15%15\\%of complete chosen–rejected pairs coincide\.

Figure 8:Matched 1\.5B pair geometry: tunedw=0\.005w=0\.005is mass\-dominated, the unit sum feasibility\-dominated; the pre\-specifiedw≈0\.1w\\approx 0\.1passes near the gate\.
#### Across\-rule contrast alignment\.

For the five scalar rules trained with the same 1\.5B recipe, define the bottleneck contrast relative to the hard gate as

b⁡\(r\)=min⁡\{Δ​FrΔ​Fgate,Δ​MrΔ​Mgate\},\(Δ​Fgate,Δ​Mgate\)=\(0\.481,0\.628\)\.b\(r\)=\\min\\\!\\left\\\{\\frac\{\\Delta F\_\{r\}\}\{\\Delta F\_\{\\rm gate\}\},\\frac\{\\Delta M\_\{r\}\}\{\\Delta M\_\{\\rm gate\}\}\\right\\\},\\qquad\(\\Delta F\_\{\\rm gate\},\\Delta M\_\{\\rm gate\}\)=\(0\.481,0\.628\)\.The quantities below use only the already reported pair\-content and endpoint tables:

scalar ruleb⁡\(r\)pair overlap with gatecomposed @16unit sum0\.1740\.0990\.596validation\-tuned sum0\.2620\.2100\.571contrast\-matched sum​\(w=0\.1\)0\.8280\.1500\.770log\-probability0\.2870\.1400\.675mass\-only0\.0710\.2620\.519\\begin\{array\}\[\]\{lccc\}\\hline\\cr\\hline\\cr\\text\{scalar rule\}&b\(r\)&\\text\{pair overlap with gate\}&\\text\{composed @16\}\\\\ \\hline\\cr\\text\{unit sum\}&0\.174&0\.099&0\.596\\\\ \\text\{validation\-tuned sum\}&0\.262&0\.210&0\.571\\\\ \\text\{contrast\-matched sum \}\(w=0\.1\)&0\.828&0\.150&0\.770\\\\ \\text\{log\-probability\}&0\.287&0\.140&0\.675\\\\ \\text\{mass\-only\}&0\.071&0\.262&0\.519\\\\ \\hline\\cr\\hline\\cr\\end\{array\}Across these five points, Spearman correlation with composed @16 is0\.900\.90forb⁡\(r\)b\(r\)and−0\.60\-0\.60for exact pair overlap\. Withn=5n=5this is deliberately presented as a descriptive ordering check, not a significance result or a fitted predictive model\. It nevertheless tests the claim on every scalar rule that was actually trained under the matched recipe, rather than only on the successfulw=0\.1w=0\.1arm\.

These are descriptive overlaps from the fixed training pool, not a statistical test\.

## Appendix RPerfect\-conjunct and finite\-pool controls

These are post\-hoc controls on existing candidates, not part of the original preregistration\. The exact feasibility oracle \(engine\) and boxed\-format regex \(GSM8K\) replace only the head of the corresponding conjunct for AUC measurement\. A binary perfect conjunct head ties every successful candidate with every failure that satisfies that conjunct\. Thus its full\-target AUC is1−q/21\-q/2, with half credit for ties\. We average over requirements containing both target classes; engine candidates are parsed\-only\. Support is285285base requirements and658,657,593658,657,593for the three matched 1\.5B sum\-DPO seeds\. Their perfect\-head AUCs are0\.83260\.8326and0\.5187,0\.5145,0\.51130\.5187,0\.5145,0\.5113, respectively\. GSM8K support is474474at base and431,443,439431,443,439after sum\-DPO, with AUCs0\.92190\.9219and0\.5189,0\.5210,0\.51550\.5189,0\.5210,0\.5155\. Own\-task oracle correctness is exact by construction; this is not an oracle for the full conjunction\.

We also measure best\-of\-kkaveraged over every uniform size\-kksubset of each stored GSM8K pool ofN=64N=64, without replacement\. For a deterministic selector order, candidate rankr∈\{1,…,N\}r\\in\\\{1,\\ldots,N\\\}is selected with probability\(N−rk−1\)/\(Nk\)\\binom\{N\-r\}\{k\-1\}/\\binom\{N\}\{k\}; averaging its target label with these weights gives the exact conditional answer rate\. Supply is1−\(N−Pk\)/\(Nk\)1\-\\binom\{N\-P\}\{k\}/\\binom\{N\}\{k\}, wherePPis the number of successes\. Ties use original candidate index\. Exhaustive small\-pool subset enumeration verifies the implementation\. This is a finite\-pool estimator, not a new scaling law or a prediction beyondNN; related ROC\-based scaling theory is in[Dorner et al\. \(2026\)](https://arxiv.org/html/2610.29029#bib.bib36)\.

For sum\-DPO, averaging training seeds, raw\-sum answers atk=4,16,64k=4,16,64are0\.693,0\.680,0\.6450\.693,0\.680,0\.645, versus base0\.606,0\.736,0\.7620\.606,0\.736,0\.762\. Supplies after sum\-DPO are0\.839,0\.929,0\.9690\.839,0\.929,0\.969at those budgets\. Replacing the learned format gate with the exact boxed regex and retaining the learned correctness ranker gives0\.6790\.679atk=16k=16and0\.6390\.639atk=64k=64after sum\-DPO \(base0\.737,0\.7580\.737,0\.758\)\. This control removes format\-classification error without repairing the remaining ranker\. It does not establish impossibility for other correctness models or selection methods\.

Finally, a fixed scale gridT∈\{0\.01,0\.1,1,10,100\}T\\in\\\{0\.01,0\.1,1,10,100\\\}applied to feasibility logits preserves every head AUC but changes raw\-sum answers\. Matched 1\.5B sum\-DPO has mean answers0\.544,0\.312,0\.185,0\.162,0\.1600\.544,0\.312,0\.185,0\.162,0\.160, respectively\. This test\-set sensitivity analysis illustrates the distinction between ranking metrics and aggregation units; it is not validation\-based hyperparameter tuning\. Code and complete tables are inscripts/audit\_perfect\_head\.py,scripts/audit\_gsm\_tails\.py, and the corresponding results directories\.

The paired requirement\-bootstrap difference \(sum\-DPO minus base\), conditional on these fixed training seeds, is−0\.0555\-0\.0555\[−0\.0738,−0\.0368\-0\.0738,\-0\.0368\] for raw\-sum answers atk=16k=16and−0\.1167\-0\.1167\[−0\.1500,−0\.0833\-0\.1500,\-0\.0833\] atk=64k=64\. Exact\-format\-gate differences are−0\.0578\-0\.0578\[−0\.0767,−0\.0402\-0\.0767,\-0\.0402\] and−0\.1193\-0\.1193\[−0\.1533,−0\.0873\-0\.1533,\-0\.0873\]\. These intervals quantify requirement uncertainty, not population training\-seed uncertainty\.

Figure 9:Post\-hoc controls\. Left: full\-target AUC for learned versus perfect own\-conjunct heads on the same pools\. Right: GSM8K finite\-pool subset\-averaged answers; exact format gating does not repair the remaining correctness ranker\. Bands show training\-seed SD, not confidence intervals\.
## Appendix STransfer forecast of on\-policy AUC

Equation[3](https://arxiv.org/html/2610.29029#S4.E3)is exact on the pool whose terms it uses, so reconstructing an AUC there is arithmetic rather than evidence\. This control asks the out\-of\-sample question instead\. The bin termsAownA\_\{\\rm own\}andAcrossA\_\{\\rm cross\}are measured on a*source*pool; from a*target*pool we read onlyqqand never the head’s ordering; we then predict the target’s on\-policy AUC\. Every arm answers the same committed test split, so requirementiidenotes the same requirement in both pools: a prediction is formed per requirement and averaged over the requirements that contain both target classes in both pools\. We compare four predictors: chance \(12\\tfrac\{1\}\{2\}\);*persistence*, the source pool’s own AUC, which is what assuming an earlier audit of the same verifier still describes the policy amounts to; the zero\-parameter form\(1−q\)​Aown\+q/2\(1\-q\)A\_\{\\rm own\}\+q/2; and the transported form\(1−q\)​Aown\+q​Across\(1\-q\)A\_\{\\rm own\}\+qA\_\{\\rm cross\}\.

Five protocols supply5757source–target pairs: the 1\.5B base pool to2626trained 1\.5B arms \(matched sum and hard\-gate DPO, the earlier DPO arms, the five IPO/DPO mitigation arms, both GRPO rewards and the second round\), the 7B base pool to99trained 7B arms, the 1\.5B base pool to those same997B arms, the 1\.5B base pool to77cVAE\-family pools scored by the same critic, and the GSM8K base pool to its66trained arms\. Measured arm AUC spans0\.4080\.408to0\.9370\.937\.

#### Why ordinary checkpoints are not a designed two\-mixture audit\.

Suppose an auditor observes\(A1,q1\)\(A\_\{1\},q\_\{1\}\)and\(A2,q2\)\(A\_\{2\},q\_\{2\}\)but the conditional terms drift from\(a1,c1\)\(a\_\{1\},c\_\{1\}\)to\(a2,c2\)\(a\_\{2\},c\_\{2\}\)\. If it nevertheless applies the fixed\-component inversiona^=\(q2​A1−q1​A2\)/\(q2−q1\)\\hat\{a\}=\(q\_\{2\}A\_\{1\}\-q\_\{1\}A\_\{2\}\)/\(q\_\{2\}\-q\_\{1\}\), then relative to the first pool

a^−a1=q1​\(1−q2\)​\(a1−a2\)\+q1​q2​\(c1−c2\)q2−q1\.\\hat\{a\}\-a\_\{1\}=\\frac\{q\_\{1\}\(1\-q\_\{2\}\)\(a\_\{1\}\-a\_\{2\}\)\+q\_\{1\}q\_\{2\}\(c\_\{1\}\-c\_\{2\}\)\}\{q\_\{2\}\-q\_\{1\}\}\.Thus separation in mixture weights cannot remove conditional drift; it can amplify it when\|q2−q1\|\|q\_\{2\}\-q\_\{1\}\|is small\. Proposition[3](https://arxiv.org/html/2610.29029#Thmproposition3)\(ii\) is therefore a statement about*controlled reweighting/resampling of fixed bin\-conditional score distributions*\. The transfer experiments below ask how far that assumption approximately travels across real policy changes; they are diagnostics, not the designed\-audit guarantee\.

Figure 10:Forecast against measurement for all5757target arms\. Each point is one target policy or pool\. The zero\-parameter form tracks the diagonal across the measured range; persistence sits in a band set by the source audit regardless of the target; chance is correct only in the saturated cluster\.Table 16:Arm\-level forecast error over all5757target arms: absolute error of the arm’s mean predicted AUC against its mean measured AUC\. The bootstrap resamples requirements jointly across arms and holds the training seeds fixed; it is requirement uncertainty, not population variance over training runs\.Table 17:Arm\-level forecast MAE by protocol\. Chance is accurate exactly where the target has saturated its own conjunct, and fails on the cVAE pools, which have not\.The1111target arms whose own conjunct has not saturated \(meanq<0\.9q<0\.9: the seven cVAE pools, two low\-β\\beta/low\-τ\\taumitigation arms and two earlier 7B DPO arms\) carry the discriminating evidence, because a chance forecast cannot be right there by construction\. Their measured AUC spans0\.5080\.508to0\.9370\.937, and the zero\-parameter MAE is0\.04290\.0429\(worst arm0\.06130\.0613\) against0\.28630\.2863for chance and0\.15960\.1596for persistence\. Transporting the measured cross term gives0\.04040\.0404\.

A further post\-hoc baseline makes the interpretation of the full5757comparisons more conservative\. The perfect\-head predictor1−q/21\-q/2has MAE0\.02610\.0261, and a leave\-one\-arm\-out linear regression of AUC onqqhas MAE0\.02610\.0261, both slightly below the source\-own forecast’s0\.02800\.0280\. Deduplicating the repeated 7B targets leaves4848unique target pools and gives the same conclusion \(0\.02610\.0261and0\.02590\.0259\)\. Thus the stored\-arm analysis demonstrates the composition relation over a wide policy range, but does not show incremental predictive value from source\-bin terms beyondqqalone\. That incremental comparison is reserved for the later arms below\.

#### What this does and does not establish\.

The forecast avoids re\-auditing the head’s ordering on the target policy; it does not avoid target\-side labels, sinceqqis an oracle\-label statistic of the target pool\. All arms existed before this control was written, so this is a post\-hoc transfer protocol across stored pools, not a preregistered forward forecast of an unrun experiment\. TransportingAownA\_\{\\rm own\}assumes the head’s ordering against own\-conjunct failures is stable across policies; the residual errors in Table[16](https://arxiv.org/html/2610.29029#A19.T16)bound how far that assumption travels here and do not license it for arbitrary policies, critics or tasks\. Accuracy on the saturated arms is unimpressive on its own, since chance achieves it\. Code:scripts/predict\_auc\_transfer\.py; complete per\-arm tables inresults/predictive\_audit/\.

#### Arms trained after the forecast was frozen\.

The forecast code and the 1\.5B base pool’s bin terms were fixed on the day the control was written; the nine strong\-controls arms \(Appendix[O](https://arxiv.org/html/2610.29029#A15)\) and the three three\-conjunct arms \(Appendix[X](https://arxiv.org/html/2610.29029#A24)\) were trained afterwards, with predictions registered before any term was computed on their pools \(PLAN\_NEW\_ARMS\_FORECAST\_2026\-09\-19\.md\)\. What remains post\-hoc is that their outcome rates had been read, so the rough location ofqtq\_\{t\}could be anticipated; that anticipation is written in the plan\. We therefore call this a pre\-specified transfer test with frozen source terms, not a fully prospective forecast\. Table[18](https://arxiv.org/html/2610.29029#A19.T18)gives the result\. The registered arm\-level error of0\.0300\.030\(band≤0\.05\\leq 0\.05\) was met \(0\.0200\.020\)\. Two post\-hoc baselines address how much of this is merelyqq: the perfect\-head form1−q/21\-q/2gives MAE0\.0300\.030, and a linear AUC–qqmodel fitted once on the earlier5757comparisons gives0\.0250\.025\. On the six non\-saturated arms the corresponding errors are0\.0160\.016,0\.0360\.036and0\.0270\.027\. A post\-hoc two\-level bootstrap resamples the1 0001\\,000committed test requirements jointly across arms and, within each of the four training rules, resamples the three observed training seeds\. It gives source\-own advantages over1−q/21\-q/2of0\.00980\.0098\[0\.0046,0\.0139\]\[0\.0046,0\.0139\]and over the historical linear baseline of0\.00540\.0054\[0\.0005,0\.0096\]\[0\.0005,0\.0096\]\. On the six non\-saturated arms the corresponding gaps are0\.01930\.0193\[0\.0092,0\.0271\]\[0\.0092,0\.0271\]and0\.01000\.0100\[0\.0006,0\.0177\]\[0\.0006,0\.0177\]\. A requirement\-only bootstrap with arms/seeds fixed gives the slightly narrower intervals reported in the artifact\. These are empirical uncertainty checks over the observed seeds and requirements, not population intervals over unseen model families or training recipes\. Averaged within training\-rule families, source\-own improves on1−q/21\-q/2in all four families and on the linear baseline in three of four; the three\-conjunct family is essentially tied with the latter\. On that subset persistence was slightly better than the registered “\>0\.10\>0\.10” \(0\.0930\.093\), a miss that leaves the ranking of the registered predictors unchanged\. Spearman correlation between measured AUC andqqover the fifteen single\-gate 1\.5B arms is−0\.82\-0\.82;AownA\_\{\\rm own\}lies in\[0\.92,0\.98\]\[0\.92,0\.98\]on every one of the eighteen arms, and the three\-conjunct training leaves the feasibility head’s terms where the two\-conjunct gate had them \(qq0\.9460\.946against0\.9160\.916\)\. Artifact:results/new\_arms\_audit/; codescripts/audit\_new\_arms\.py\.

Table 18:Pre\-specified transfer test on the twelve arms trained after the source terms were frozen \(means of three seeds; 1\.5B base as source\)\.qtq\_\{t\}, measured AUC and the forecast are over the requirements whose pools support the terms in both source and target;AownA\_\{\\rm own\}is the target arm’s own, shown for comparison and not used\. “Forecast” is the zero\-parameter form\(1−qt\)​Aownsrc\+qt/2\(1\-q\_\{t\}\)A\_\{\\rm own\}^\{\\rm src\}\+q\_\{t\}/2\.
#### Along a run\.

The within\-run version, registered before any checkpoint pool was generated \(PLAN\_WITHIN\_RUN\_FORECAST\_2026\-09\-19\.md\), forecasts the head’s on\-policy AUC at checkpointk\+1k\{\+\}1from the bin terms at checkpointkk, exactly as across arms\. Four seed\-0 runs with adapters saved every250250steps \(log\-probability, tuned sum, three\-conjunct gate, mass\-only\) were sampled on the first250250committed test requirements atK=16K\{=\}16; step00and step1 5001\\,500are the corresponding rows of the stored base and final pools\. Table[19](https://arxiv.org/html/2610.29029#A19.T19)and Figure[11](https://arxiv.org/html/2610.29029#A19.F11)give the result\. Over the1818transitions of the three arms whoseqqmoves, the zero\-parameter forecast has arm\-level error0\.0490\.049\(registered0\.030\.03, band≤0\.06\\leq 0\.06\) against0\.0960\.096for persistence and0\.1380\.138for chance, and beats persistence on1515of1818; from the base pool’s terms alone the error over every later checkpoint is0\.0320\.032\. That per\-requirement comparison is confined to the requirements where the source checkpoint still has both negative bins, which on a saturated checkpoint are its least saturated ones; applying the source’s pooled own term to every target requirement instead \(the arm\-level form used out of distribution, and what Figure[11](https://arxiv.org/html/2610.29029#A19.F11)plots\) gives0\.0210\.021against0\.0640\.064for persistence and0\.0800\.080for chance,1313of1818transitions in the forecast’s favour, worst transition0\.0640\.064\. Saturation is fast: on the log\-probability and three\-conjunct runsqqexceeds0\.90\.9by step500500\(registered\), the first transition is persistence’s worst, and the head’s AUC has already fallen to0\.540\.54and0\.490\.49at step250250while per\-draw success is0\.090\.09and0\.160\.16\. The mass\-only run is the registered null: itsqqstays in\[0\.28,0\.41\]\[0\.28,0\.41\], persistence is within0\.0300\.030, and the forecast gains only0\.0100\.010on it — the forecast wins whereqqmoves and only there\. Two registered checks missed:Aown≥0\.90A\_\{\\rm own\}\\geq 0\.90at every checkpoint fails once \(0\.8530\.853, log\-probability step1 0001\\,000, on a saturated pool with few own\-bin pairs among250250requirements\), and the tuned sum’sqqreached0\.4980\.498rather than\>0\.5\>0\.5by step500500\. Single seed,250250requirements, and generation batches that differ between the checkpoint pools and the stored endpoints are the limits of this control\. Artifact:results/within\_run\_forecast/; codescripts/forecast\_within\_run\.py\.

Table 19:Within\-run forecast from the previous checkpoint: arm\-level mean absolute error of each predictor over the six transitions of each run \(250250requirements,K=16K\{=\}16, seed 0\)\. The mass\-only run is the registered null case whoseqqdoes not move\.Figure 11:Measured on\-policy AUC of the feasibility head \(dots\),qq\(squares\) and the zero\-parameter forecast from the previous checkpoint’s pooled own term \(crosses\) along four DPO runs;250250requirements,K=16K\{=\}16, seed 0\.
#### Out of distribution\.

The benchmark ships a fourth split of500500requirements drawn beyond the training prior \(thrust to61\.861\.8N against a55–6060N prior, expansion ratio to23\.323\.3\)\. Fifteen stored cVAE pools answer it\. Requirement indices do not correspond across splits, so this forecast is arm\-level: each generator family’s mean bin terms from the test split, that family’s meanqqon the OOD split, and its OOD AUC predicted\. Over the five families the zero\-parameter mean absolute error is0\.02820\.0282, against0\.07520\.0752for persistence and0\.36400\.3640for chance, and the identity still reconstructs every OOD requirement to10−910^\{\-9\}\. Four families land within0\.0240\.024\. The capped decoder is the exception, missing by0\.1010\.101, and the terms say why: its own\-bin ordering does not transport, falling fromAown=0\.965A\_\{\\rm own\}=0\.965on the test split to0\.8210\.821out of distribution\. That is the assumption the forecast rests on and this arm is where it breaks\. The transported variant fares worse still on latent search, whoseAcrossA\_\{\\rm cross\}drops from0\.8170\.817to0\.6510\.651\. Artifact:results/ood\_split/audit\.json; codescripts/audit\_ood\_split\.py\.

## Appendix TA label\-free version of the diagnostic does not transfer

Every measurement above needs oracle labels:qq,AownA\_\{\\rm own\}andAcrossA\_\{\\rm cross\}are all label statistics\. A deployed pipeline has none, so we asked whether the critic’s own scores already reveal that an additive composition is losing best\-of\-KKlift\. Three label\-free statistics were computed per arm from the two term arrays alone: the share of pools whose sum\-argmax equals the saturated term’s argmax, the median within\-pool SD of the saturated term over the other term’s, and the share of candidates the saturated head calls satisfied withp\>0\.99p\>0\.99\. They were asked to predict two oracle\-measured quantities:ηraw\\eta\_\{\\rm raw\}, the share of attainable best\-of\-KKlift the additive rule captures, and the gain from switching to a gate\. Leave\-one\-arm\-out linear fits were compared against predicting the training\-fold mean over3232engine arms and the same3232plus77GSM8K arms\.

On the engine arms alone the relations look usable: saturated fraction against gate gain reachesr=0\.70r=0\.70with leave\-one\-out MAE0\.1400\.140against0\.2270\.227for the mean, and the SD ratio againstηraw\\eta\_\{\\rm raw\}reachesr=0\.73r=0\.73with MAE0\.1120\.112against0\.1510\.151\. The SD ratio’s sign is nevertheless opposite to the mechanism, because a dominant feasibility term also marks pools where feasibility genuinely discriminates\. Adding GSM8K removes the effect: saturated fraction against gate gain falls tor=0\.02r=0\.02and the SD ratio tor=−0\.14r=\-0\.14, both worse than predicting the mean \(0\.3390\.339and0\.3360\.336against0\.3320\.332\)\.

We therefore make no label\-free claim\. This is a negative result on three specific statistics and two targets, not a proof that no label\-free diagnostic exists; a learned combination, per\-requirement rather than per\-arm targets, or a calibrated head might do better\. Code:scripts/label\_free\_diagnostic\.py; full table inresults/label\_free\_diagnostic/audit\.json\.

## Appendix UFeasibility’s own sub\-constraints: a case where the mix moves and the AUC does not

The packaged oracle definesfeasibleasprintable AND fits\_envelope: no DfAM flag \(overhang, minimum wall, channel\-versus\-ligament, minimum TPMS cell\) and length≤\\leqenvelope \(the radius clause2​rmax≤2r\_\{\\max\}\\leqenvelope cannot bind:2×55​mm<200​mm2\\times 55\\,\\mathrm\{mm\}<200\\,\\mathrm\{mm\}\)\. Both are consequential geometric constraints, and neither is syntactic\. The main text treats feasibility as one conjunct; here we split it\.

The two are learned at very different rates\. Printability rises from0\.3960\.396at the 1\.5B base and0\.5440\.544at the 7B base to0\.930\.93–1\.001\.00across every matched DPO and GRPO arm; envelope fit starts at0\.8080\.808in both bases and ends between0\.8110\.811and0\.8800\.880\. The envelope rate is capped: for4\.9%4\.9\\%of test requirements the sized length exceeds the envelope even at the policy’s shortest setting, so no design fits\. Among attainable requirements the fit rate moves from0\.850\.85to0\.920\.92under sum\-ranked pairs and stays at0\.860\.86–0\.870\.87under gated pairs \(Appendix[W](https://arxiv.org/html/2610.29029#A23)\)\. The composition of the head’s negative bin therefore shifts hard: writingp=Pr⁡\[printable∣not feasible\]p=\\Pr\[\\text\{printable\}\\mid\\text\{not feasible\}\], the share of remaining failures that print but do not fit rises from0\.0810\.081\(1\.5B base\) and0\.0520\.052\(7B base\) to0\.210\.21–0\.930\.93after training\.

Splitting the feasibility head’s negatives the same way givesAUC⁡\(sF,F\)=\(1−p\)​Aprint\+p​Aenv\\mathrm\{AUC\}\(s\_\{F\};F\)=\(1\-p\)A\_\{\\rm print\}\+p\\,A\_\{\\rm env\}, checked to10−910^\{\-9\}per requirement\. Unlike the conjunctive case of the main text, the inherited term is not near chance:AenvA\_\{\\rm env\}lies between0\.8890\.889and0\.9950\.995on every arm, andAUC⁡\(sF,F\)\\mathrm\{AUC\}\(s\_\{F\};F\)stays between0\.9400\.940and0\.9910\.991throughout\. The extreme case is the 7B sum arm at seed 2, wherep=0\.932p=0\.932— almost every remaining failure is an envelope failure — and the AUC is0\.9830\.983\.

This is the control the conjunctive claim needs\. Negative composition moves at least as far here as it does in the main text \(qqreaching0\.980\.98\), in the same pools and with the same critic, and no collapse follows, because the head kept its ordering in the bin it inherited\. Composition and bin quality are separately necessary; neither on its own predicts the diagnostic\. The envelope\-only bin is sparse on the strongest arms \(3333–201201supported requirements, listed per arm in the artifact\), so theAenvA\_\{\\rm env\}estimates there carry correspondingly wide uncertainty; the direction of the contrast does not depend on any single arm\. Code:scripts/audit\_feasibility\_substructure\.py; per\-arm table inresults/subconstraint\_audit/audit\.json\.

## Appendix VThe same critic as a training target and as a search objective

The cVAE family lets the critic enter the generator in two different places, on one design space with one oracle\. Five families, three decoder seeds each,1 0001\\,000test requirements andK=64K=64; mean±\\pmseed SD\.

Table 20:Generator families on the engine task\. Per\-draw on\-target is flat across the first three families while supply collapses, which is the dissociation of §[AE](https://arxiv.org/html/2610.29029#A31)in the generator\-training direction rather than the selection direction\. Latent search is not compute\-matched with the decoders\.Training the decoder against the critic’s two terms leaves per\-draw on\-target where the untrained prior already was and removes two thirds of the coverage; capping the output layer, the obvious fix for the escape of Appendix[F](https://arxiv.org/html/2610.29029#A6), does not recover it and in these runs is worse\. Theλ=1\\lambda=1rows carry very large seed spread — one seed reaches feasible0\.2380\.238and another0\.0490\.049— so we read the supply collapse, which is consistent across seeds, and not the feasible column\. Using the same critic as a search objective over the same decoder’s latent space instead reaches0\.3420\.342on target and0\.9370\.937supply\.

We do not claim an optimality result for latent search, which is not compute\-matched, and we do not claim that no decoder training against a critic can work\. The reading we do take is that a critic which ranks candidates well \(Table[2](https://arxiv.org/html/2610.29029#S4.T2)\) is not thereby a good objective for shaping the generator that produces them, and that the failure shows up as lost coverage rather than lost per\-draw quality — the same place it shows up in the language\-model arms\. The ordering survives the out\-of\-distribution split: supply atK=64K\{=\}64falls to0\.2560\.256for the prior and0\.2550\.255forλ=0\\lambda=0, stays at0\.0950\.095and0\.0830\.083for the two critic\-trained decoders — which have no coverage left to lose — and falls to0\.5960\.596for latent search, which degrades most in relative terms and still dominates\. Artifacts:results/generator\_families/audit\.jsonandresults/ood\_split/audit\.json\.

## Appendix WWhat the generator learns to condition on the requirement

The engine target has one conjunct that does not depend on the requirement and two that do\.*Printability*is a set of DfAM rules on walls, ligaments, overhang and cell size; the requirement enters only through the bore radii\.*Envelope fit*is, from the oracle code,L⁡\(thrust,ε\)⋅\(0\.85\+0\.4​v30\)≤envelopeL\(\\text\{thrust\},\\varepsilon\)\\cdot\(0\.85\+0\.4\\,v\_\{30\}\)\\leq\\text\{envelope\}: one design coordinate, the length scalev30v\_\{30\}, against a sized length and a per\-prompt envelope\.*Mass on target*needs the body’s mass to track the prompt’s budgetm⋆m^\{\\star\}\. Reading the rule off the code and re\-applying it to stored candidates reproduces the oracle’s envelope flag on100\.0%100\.0\\%of parsed candidates on all3939pools\. Table[21](https://arxiv.org/html/2610.29029#A23.T21)measures, per generator, whether each conjunct is satisfied as a habit or as a function of the prompt: the partial correlation of the requirement\-meanv30v\_\{30\}withlog\\logenvelope givenlog\\logsized length, and the OLS gain of requirement\-meanlog⁡m\\log monlog⁡m⋆\\log m^\{\\star\}\. Every quantity is behavioural, measured on outputs; nothing here inspects a representation inside a model\.

Table 21:What each generator conditions on the requirement\.*Printable*is the requirement\-independent conjunct; its rate and its spread across thrust terciles \(pp\)\.*Envelope*is the requirement\-dependent conjunct with one control, the length scalev30v\_\{30\}: partial correlation of the requirement\-meanv30v\_\{30\}withlog\\logenvelope givenlog\\logsized length \(reads the envelope?\), and the fit rate where fitting is attainable at all \(95\.1%95\.1\\%of requirements\)\.*Mass*: OLS slope of requirement\-meanlog⁡m\\log monlog⁡m⋆\\log m^\{\\star\}\(a gain;11is one\-to\-one tracking\) and its partial correlation given thrust\. Test split, mean±\\pmSD over three seeds; single\-seed rows unmarked\. LLM armsK=16K\{=\}16, cVAEK=64K\{=\}64\.†Arms trained after the audit was written, with predictions registered beforehand\.#### The requirement\-independent conjunct goes to ceiling under every rule that contrasts it\.

Printability is0\.400\.40and0\.540\.54at the two bases and0\.930\.93–1\.001\.00on every DPO and GRPO arm whose pairs contrast feasibility; its dependence on thrust,4242and2222pp across terciles at the bases, falls to00–55pp on those arms\. This is the conjunct whose head saturates in the main text\. The two rules whose pairs barely contrast feasibility \(Table[14](https://arxiv.org/html/2610.29029#A15.T14)\) leave it short of ceiling:0\.57±0\.070\.57\\pm 0\.07under mass\-only, whose pairs contrast it by0\.030\.03, and0\.80±0\.030\.80\\pm 0\.03under the tuned sum \(0\.130\.13\), with the thrust dependence of the base intact \(3535–3636pp\)\. We had registered the mass\-only value as0\.600\.60\(band<0\.80<0\.80\) before computing it\.

#### Which requirement\-dependent conjunct is conditioned on is set by the ranking rule\.

Pairs ranked by the unit\-weight sum, which the saturated feasibility logit dominates, install envelope conditioning: the partial correlation ofv30v\_\{30\}with the envelope rises from0\.220\.22to0\.760\.76–0\.840\.84at 1\.5B and from0\.630\.63to0\.910\.91–0\.940\.94at 7B, on every seed, while the mass gain reaches only0\.440\.44–0\.520\.52at 1\.5B and*falls*from the 7B base’s0\.300\.30to0\.190\.19–0\.220\.22\. Pairs ranked by the hard gate leave envelope conditioning where the base had it \(0\.120\.12–0\.160\.16at 1\.5B;0\.480\.48–0\.620\.62at 7B\) and install mass conditioning with near\-unit gain:1\.03±0\.081\.03\\pm 0\.08at 1\.5B with partial correlation0\.970\.97on every seed,0\.62±0\.060\.62\\pm 0\.06at 7B\. At 1\.5B the across\-requirement SD of the gated policy’s meanlog⁡m\\log mis0\.660\.66, the SD of the log\-uniform budget prior itself being0\.610\.61: the policy’s mass distribution has become the budget distribution\. Per\-draw on\-target tracks mass gain, the quantity on\-target requires: Spearman0\.960\.96over the seven trained LLM families and0\.880\.88with the two bases, whose printability is still far from ceiling, included\. The two GRPO rewards, whose effect on supply and per\-draw is small in §[AF](https://arxiv.org/html/2610.29029#A32), are indistinguishable here as well \(0\.680\.68vs0\.600\.60envelope,0\.200\.20vs0\.210\.21mass\)\. Both DPO rules still varyv30v\_\{30\}within a requirement \(SD0\.040\.04–0\.070\.07\); “not conditioned” means not set by the prompt, not held constant\.

#### Scalar rules that contrast mass install partial tracking; the gates install full tracking\.

The four arms trained after this audit was written \(†\\daggerin Table[21](https://arxiv.org/html/2610.29029#A23.T21)\) test the rule\-chooses\-the\-conjunct claim prospectively\. On envelope they behave as registered: the tuned sum and mass\-only leave it where the base had it \(0\.080\.08,0\.110\.11\), the log\-probability form installs it like the sum \(0\.790\.79\), and the three\-conjunct gate leaves it like the gate \(0\.110\.11\)\. On mass they do not: we registered gains of0\.950\.95for the tuned sum and mass\-only \(band\[0\.8,1\.15\]\[0\.8,1\.15\]\) and0\.450\.45for log\-prob \(band\[0\.3,0\.65\]\[0\.3,0\.65\]\), and all three land at0\.720\.72–0\.740\.74, whatever their pairs’ mass contrast \(0\.180\.18for log\-prob,0\.810\.81–0\.840\.84for the other two\); the three\-conjunct gate lands at1\.061\.06, as registered\. Across the five single\-gate rules the trained printable rate follows the pairs’ feasibility contrast \(Spearman0\.90\.9\) but mass gain does not follow the mass contrast \(0\.60\.6\): both gates reach unit gain with an across\-requirement mass SD of0\.660\.66–0\.680\.68that matches the budget prior’s0\.610\.61, while these scalar rules stop at0\.440\.44–0\.500\.50\. The contrast\-matched sum of Appendix[O](https://arxiv.org/html/2610.29029#A15)settles what this means: atw=0\.1w=0\.1a scalar score installs unit budget tracking \(0\.98±0\.020\.98\\pm 0\.02, mass SD0\.640\.64\)*and*envelope conditioning \(0\.69±0\.060\.69\\pm 0\.06\), the only rule that installs both\. The plateau belonged to the wrong weights, not to scalar scores\.

#### The continuous decoder takes the unconditional shortcut; search does not\.

No cVAE decoder conditions mass on the budget \(gain−0\.01\-0\.01–0\.000\.00on all nine seeds, prior included\), although the decoder is conditional by construction\. Trained against the critic’s terms, theλ=1\\lambda=1and capped decoders satisfy the envelope by collapsingv30v\_\{30\}to0\.000±0\.0010\.000\\pm 0\.001on every seed — the shortest allowed length for every prompt — reaching fit1\.0001\.000while printability falls to0\.000\.00–0\.250\.25\. Latent search over the untrained decoder, under the same critic, attains mass gain0\.760\.76with partial correlation0\.990\.99: in the continuous representation the prompt dependence arrives through search at inference, not through the learned map\. This is the same contrast as Table[20](https://arxiv.org/html/2610.29029#A22.T20), one level down: the critic that shapes a token policy’s conditioning through pair ranking shapes a decoder into a constant\.

#### Registered predictions and misses\.

Five predictions were written before the audit \(context/docs/PLAN\_CONDITIONING\_2026\-09\-18\.md\); each part is scored there\. Held: sum gain below gate gain at both scales; printability flat across envelope classes; gated arms spread mass across requirements more than sum arms\. Missed, in the informative direction: we predicted that trained token policies would take the decoders’ shortcut \(global shrink,\|ρ\|<0\.15\|\\rho\|<0\.15\) — they condition instead, on the conjunct the rule rewards; that gated mass gain would lie in0\.300\.30–0\.700\.70— at 1\.5B it is1\.031\.03; that the base 7B model would not read the budget — it does, partially \(0\.300\.30\); and that sum arms would narrow their mass below the base’s spread — at 1\.5B they widen it, because they track the budget partially\. Also missed: the attainable fraction is0\.9510\.951, not0\.900\.90, and printability on the 7B gated and GRPO gated arms keeps a66–1414pp thrust dependence\. Base and single\-lineage rows are one seed and all arms pre\-existed the audit; this is a post\-hoc characterisation with pre\-registered quantities, not a prospective test\. Code:scripts/audit\_conditioning\.py, tests inscripts/test\_conditioning\.py; per\-arm values inresults/conditioning\_audit/audit\.json\.

## Appendix XA physics conjunct outside every score

Nothing in the paper’s engine target asks whether the printed body would survive a firing\. The repository’s print\-and\-fire readiness screen does: Bartz throat heat flux at the operating point the requirement implies \(pc=F/\(CF​At\)=1\.30p\_\{c\}=F/\(C\_\{F\}A\_\{t\}\)=1\.30bar for every requirement, sincert∝Fr\_\{t\}\\propto\\sqrt\{F\};ϕ=1\\phi=1,Tc=2287T\_\{c\}=2287K,c∗=1249c^\{\*\}=1249m/s, within1\.1%1\.1\\%of the repository’s Cantera values\); a closed\-form steady hot face through the liner, cooled by regenerative air at 9 bar through the body’s own cooling\-zone channels \(film coefficient from the air velocity in the channels that fit mid\-wall\); von Mises margin against the AlSi10Mg yield at the hot\-face temperature; and a Coffin–Manson low\-cycle\-fatigue life of at least2020fires\. It is closed\-form,0\.140\.14ms per candidate, and we applied it to every stored candidate in all3939pools \(Table[22](https://arxiv.org/html/2610.29029#A24.T22)\)\. Ignition is excluded becauseL∗L^\{\*\}is fixed by the oracle’s sizing and the verdict is constant\. Neither the critic nor any policy was trained on any of this; the critic scores printability, envelope and mass\.

Table 22:A physics conjunct no score has seen\. Every stored candidate is re\-graded by the readiness screen \(hot\-face temperature of the AlSi10Mg liner under Bartz throat flux, cooled by regenerative air through its own channels; yield margin at temperature; Coffin–Manson life≥20\\geq 20fires\) at the operating point the requirement implies \(pc=1\.30p\_\{c\}=1\.30bar,ϕ=1\\phi=1\)\.AcoolA\_\{\\rm cool\}: mean total cooling\-channel flow area \(mm2\), which sets the coolant velocity and hence the hot face\.*ready*∣\\mid*feasible*: survival among feasible bodies\. Supply and two\-stage answer are given for the original target \(on target\) and the extended one \(on target∧\\wedgeready\), with the screen’s selection efficiencyηext\\eta\_\{\\rm ext\}and the efficiency of the same rule if it could also see the readiness verdict\. Mean±\\pmSD over seeds;K=16K\{=\}16\(LLM\) and6464\(cVAE\)\.†Arms trained after the audit was written; the three\-conjunct gate ranks pairs by feasibility∧\\wedgereadiness, then mass\.#### The conjunct binds, and its lever is the cooling channels\.

Among feasible bodies,2929–64%64\\%survive\. The hot face is set by the coolant film coefficient, i\.e\. by the total channel flow areaAcool=n​π​reff2A\_\{\\rm cool\}=n\\pi r\_\{\\rm eff\}^\{2\}: more or wider channels slow the air and heat the wall\. This is the metal that mass minimisation removes\. The 7B gated policy, whose mass tracks the budget with gain0\.620\.62\(Appendix[W](https://arxiv.org/html/2610.29029#A23)\), opens8686mm2against the sum policy’s5151mm2and runs its median hot face at604∘604^\{\\circ\}C against409∘409^\{\\circ\}C, so0\.310\.31of its feasible bodies survive against0\.540\.54\. Latent search, the best generator under the original target \(0\.3750\.375per draw, supply0\.940\.94\), opens116116mm2, runs at1114∘1114^\{\\circ\}C and survives at0\.180\.18\. The two critic\-trained decoders have no cooling zone in100%100\\%of their designs — the printability rule that penalises a channel against its ligament is satisfied by deleting the channels — and zero survival\.

#### What the training rules protect is what the score ranks\.

Under the extended target, on target∧\\wedgeready, the gate’s composed advantage over the sum shrinks at 1\.5B \(0\.2960\.296against0\.2600\.260; supply0\.4440\.444against0\.3710\.371\) and inverts on two of three seeds at 7B \(0\.2030\.203against0\.2310\.231; supply0\.2490\.249against0\.2680\.268\)\. Survival among feasible bodies is higher than the base’s on every trained DPO arm \(12/1212/12\), but the ordering between rules is not stable across scales\. We had registered that the gate would keep its advantage on every seed at both scales; it does not, and the reversal at 7B is the informative miss\. The selection side is more robust: the two\-stage screen, which sees no physics, still capturesηext=0\.51\\eta\_\{\\rm ext\}=0\.51–0\.860\.86of the attainable extended lift, because feasibility and mass are necessary conditions; the same rule given the readiness verdict as one more conjunct reaches0\.870\.87–0\.970\.97\. The feasibility head’s own ordering against the physics bin is not at chance \(AphysA\_\{\\rm phys\}0\.440\.44–0\.690\.69; three\-bin identity exact to2×10−162\\times 10^\{\-16\}\), which is the wall–ligament coupling, not knowledge of heat transfer\.

#### Registered predictions and misses\.

Eight predictions were written before the audit \(context/docs/PLAN\_PHYSICS\_GATE\_2026\-09\-18\.md\)\. Held: every trained DPO arm survives better than its base; the identity is exact;AphysA\_\{\\rm phys\}on gated 1\.5B is0\.5540\.554against a registered0\.550\.55;ηext\\eta\_\{\\rm ext\}on gated 1\.5B is0\.5570\.557against0\.550\.55\. Missed: every absolute survival rate \(we predicted0\.450\.45–0\.650\.65and measured0\.290\.29–0\.410\.41; the screen is harsher on AlSi10Mg than the CuCrZr numbers we calibrated on\); gate above sum on every seed \(reversed at 7B\); supply and answer under the extended target for gated 1\.5B \(0\.4440\.444and0\.2960\.296against0\.550\.55and0\.350\.35\); latent search survival \(0\.180\.18against0\.500\.50\); and that no trained policy sets its wall from the thrust — the sum\-ranked and GRPO policies do \(partial correlation0\.300\.30–0\.630\.63\), through the bore radii that printability depends on, while the gated 1\.5B policy does not \(0\.010\.01–0\.080\.08\)\. All arms in the paragraphs above pre\-existed the audit\. Code:scripts/audit\_physics\_gate\.py; per\-arm values and the declared operating point inresults/physics\_gate\_audit/audit\.json\.

#### The policies that track mass hardest run hottest\.

The four arms trained afterwards \(†\\daggerin Table[22](https://arxiv.org/html/2610.29029#A24.T22)\) were screened with predictions registered first \(PLAN\_NEW\_ARMS\_CPU\_2026\-09\-19\.md\)\. The two rules whose pairs contrast mass and not feasibility open the widest channels \(9393and104104mm2against the gate’s5959\), run at692692and719∘719^\{\\circ\}C, and survive at0\.280\.28–0\.290\.29among feasible bodies, below the gate on every seed and at the base’s level; the log\-probability form sits with the sum \(0\.420\.42; registered within0\.050\.05of it\)\. The registered claim that every trained arm survives better than the base in absolute terms fails on two of twelve seeds \(tuned sum and mass\-only, at0\.260\.26–0\.270\.27against the base’s0\.280\.28\), because those policies never installed the feasibility that survival rides on\.

#### The gate composes conjuncts, at a price\.

Putting the readiness verdict into the gate — pairs ranked bymass\_term−100\[prob<0\.5\]−100\[¬ready\]\-100\\,\[\\texttt\{prob\}<0\.5\]\-100\\,\[\\lnot\\text\{ready\}\], the readiness label computed for the stored train pool in seconds, the recipe otherwise unchanged — and training three seeds \(PLAN\_PHYSICS\_DPO\_2026\-09\-18\.md, registered before the job\) raises survival among feasible bodies from0\.400\.40to0\.550\.55\(0\.520\.52–0\.570\.57, every seed above the two\-conjunct gate\), the extended composed answer from0\.300\.30to0\.370\.37, and the physics\-aware ceiling from0\.410\.41to0\.460\.46, with mass tracking intact \(gain1\.041\.04–1\.081\.08\) and printability at ceiling\. Of eleven registered checks seven held and four missed, all in the direction of a smaller effect: survival0\.550\.55against a registered0\.600\.60, extended per\-draw0\.1350\.135against0\.200\.20, extended supply0\.500\.50against0\.600\.60; the falsifier \(survival below0\.450\.45\) did not fire\. The miss that says most: the policy’s channel area still does not depend on the thrust \(0\.30→0\.310\.30\\to 0\.31partial correlation\), so it improved survival unconditionally — channels59→4059\\to 40mm2, median hot face485→365∘485\\to 365^\{\\circ\}C — rather than by learning the coolant requirement the thrust implies, the same pattern as the unconditional shortcut in Appendix[W](https://arxiv.org/html/2610.29029#A23)\. The price is paid on the original target: per\-draw0\.311→0\.2570\.311\\to 0\.257, supply0\.81→0\.740\.81\\to 0\.74, composed0\.76→0\.690\.76\\to 0\.69\. The feasibility head’s mixture terms are unchanged \(qq0\.9460\.946,AownA\_\{\\rm own\}0\.9580\.958\)\. Artifacts:reproduced/physics\_gate\_dpo/; codescripts/summarize\_physics\_gate\_dpo\.py\.

## Appendix YSelection ambiguity and the audit card: proofs and validation

#### Proof of Proposition[5](https://arxiv.org/html/2610.29029#Thmproposition5)\.

Let the pool be\{x1,…,xn\}\\\{x\_\{1\},\\ldots,x\_\{n\}\\\}with distinct score pairs\(fi,mi\)=\(sF​\(xi\),sM​\(xi\)\)\(f\_\{i\},m\_\{i\}\)=\(s\_\{F\}\(x\_\{i\}\),s\_\{M\}\(x\_\{i\}\)\)and letPar\\operatorname\{Par\}be the set of candidates not weakly dominated by any other\.*Only Pareto candidates can win\.*Ifxjx\_\{j\}dominatesxix\_\{i\}\(fj≥fif\_\{j\}\\geq f\_\{i\},mj≥mim\_\{j\}\\geq m\_\{i\}, one strict\), then for any strictly increasinggg,g⁡\(fj\)\+mj\>g⁡\(fi\)\+mig\(f\_\{j\}\)\+m\_\{j\}\>g\(f\_\{i\}\)\+m\_\{i\}, soxix\_\{i\}is never the top\-1 choice\.*Every Pareto candidate can win\.*Fixp∈Parp\\in\\operatorname\{Par\}\. Everyxix\_\{i\}withfi\>fpf\_\{i\}\>f\_\{p\}hasmi<mpm\_\{i\}<m\_\{p\}\(otherwise it would dominatepp\), and no other candidate hasfi=fpf\_\{i\}=f\_\{p\}withmi≥mpm\_\{i\}\\geq m\_\{p\}\. LetΔ=min\{mp−mi:fi≥fp,i≠p\}\>0\\Delta=\\min\\\{m\_\{p\}\-m\_\{i\}:f\_\{i\}\\geq f\_\{p\},\\,i\\neq p\\\}\>0\(orΔ=1\\Delta=1if the set is empty\),ρ=maxi⁡fi−mini⁡fi\\rho=\\max\_\{i\}f\_\{i\}\-\\min\_\{i\}f\_\{i\},μ=maxi⁡mi−mini⁡mi\\mu=\\max\_\{i\}m\_\{i\}\-\\min\_\{i\}m\_\{i\}, chooseε\>0\\varepsilon\>0withε​ρ<Δ\\varepsilon\\rho<\\Delta, and setg⁡\(t\)=ε​tg\(t\)=\\varepsilon tfort<fpt<f\_\{p\}andg⁡\(t\)=K\+ε​tg\(t\)=K\+\\varepsilon tfort≥fpt\\geq f\_\{p\}withK\>μ\+ε​ρK\>\\mu\+\\varepsilon\\rho\. Thenggis strictly increasing; every candidate withfi<fpf\_\{i\}<f\_\{p\}scores belowppbecauseK\>μ\+ε​ρK\>\\mu\+\\varepsilon\\rho; and every candidate withfi≥fpf\_\{i\}\\geq f\_\{p\},i≠pi\\neq p, scoresK\+ε​fi\+mi<K\+ε​fp\+mpK\+\\varepsilon f\_\{i\}\+m\_\{i\}<K\+\\varepsilon f\_\{p\}\+m\_\{p\}becauseε⁡\(fi−fp\)≤ε​ρ<Δ≤mp−mi\\varepsilon\(f\_\{i\}\-f\_\{p\}\)\\leq\\varepsilon\\rho<\\Delta\\leq m\_\{p\}\-m\_\{i\}\. Henceppis the unique top\-1 choice\.*Linear rescalings\.*Forg⁡\(t\)=c​tg\(t\)=ct,c\>0c\>0, the top\-1 choice maximizes the linear functional\(c,1\)⋅\(f,m\)\(c,1\)\\cdot\(f,m\), whose maximizers over the finite set are exactly the points of the convex hull exposed by a direction in the open positive quadrant; these are Pareto points on the upper hull\. The identification statements follow because an audit statistic invariant to monotone reparameterization is constant on the class\{g⁡\(sF\)\}\\\{g\(s\_\{F\}\)\\\}while the decision ranges overPar\\operatorname\{Par\}\.□\\square

With ties \(duplicated candidates are common when a policy repeats a program\), apply the result to distinct score pairs; tied candidates share scores and, in our tasks, oracle labels\.

#### Measured ambiguity\.

Table[23](https://arxiv.org/html/2610.29029#A25.T23)reports, per requirement pool of parsed candidates, the mean Pareto size, the share of pools whose Pareto set is a singleton, the share mixing on\- and off\-target candidates, and the per\-pool outer range of Answer@16 reachable by audit\-equivalent composites \(all monotonegg\) and by positive rescalings\. Pair\-weight\-trained Phi policies nearly eliminate ambiguity because they emit one program \(Section[AF](https://arxiv.org/html/2610.29029#A32)\)\.

Table 23:Selection ambiguity on ECG test pools \(post\-hoc\)\. Ranges are per\-pool outer bounds: a single transform must serve all pools, so the attainable aggregate range can be narrower\.
#### Audit\-card validation\.

All numbers use the frozen ECG base test pools, parsed candidates, and 12\-subject cluster resampling \(seed fixed in the released script\)\.*Identified interval \(step 1\)\.*For the deployability head, the per\-pool intervals average to\[0\.472,1\]\[0\.472,1\]for Llama \(292 pools with a positive and a negative; outer 95% interval\[0\.396,1\]\[0\.396,1\]\) and\[0\.452,1\]\[0\.452,1\]for Phi \(\[0\.389,1\]\[0\.389,1\]\), while the direct value is1\.0001\.000;37%37\\%and48%48\\%of pools are individually erased\. For the utility head,qqis small \(0\.0130\.013–0\.0180\.018\), the interval is tight \(\[0\.894,0\.900\]\[0\.894,0\.900\]and\[0\.872,0\.877\]\[0\.872,0\.877\]\), and no labelling is needed\.*Label budget \(step 2\)\.*We labelmmoff\-target candidates uniformly at random, compare each revealed own\-failure with all positives in its pool, and use the ratio estimator of the pair\-weighted within\-poolAownA\_\{\\rm own\}, with a subject\-cluster bootstrap over the labelled items \(200 repetitions permm\)\.

Table 24:Label budget for a direct own\-bin audit of the ECG utility head \(post\-hoc\)\. Target: pair\-weighted within\-poolAownA\_\{\\rm own\}with all labels \(0\.8950\.895Llama,0\.8420\.842Phi\)\.*Controlled mixtures \(step 3\)\.*Pooling the bin\-conditional score distributions, we draw 200 positives and 400 negatives at mixture weightsq1,q2q\_\{1\},q\_\{2\}, compute the two aggregate AUCs, and solveAown=\(q2​A1−q1​A2\)/\(q2−q1\)A\_\{\\rm own\}=\(q\_\{2\}A\_\{1\}\-q\_\{1\}A\_\{2\}\)/\(q\_\{2\}\-q\_\{1\}\)\. Over 200 repetitions the bias is below0\.0150\.015in all settings; RMSE is0\.0100\.010–0\.0210\.021at\(q1,q2\)=\(0\.2,0\.8\)\(q\_\{1\},q\_\{2\}\)=\(0\.2,0\.8\),0\.0340\.034–0\.0470\.047at\(0\.4,0\.6\)\(0\.4,0\.6\)and0\.0750\.075–0\.0900\.090at\(0\.45,0\.55\)\(0\.45,0\.55\)for both heads and families\. The utility head’s inherited bin is small \(25 and 57 candidates\), so its mixtures resample those candidates heavily\.

## Appendix ZPreregistered embedded\-ECG stress test

#### Purpose\.

This study was designed after the original engine results as an external stress test, not as an exploratory search for another positive domain\. The benchmark, critic, model revisions, pair builders, scalar grids, DPO recipe, primary endpoint, equivalence margin, and 30\-run matrix were frozen before confirmatory DPO\. The optional INCART OOD phase was disabled before the primary freeze and was not run\. The freeze is an internal commitment: protocol, manifest and predictions are SHA\-256 hashed in the project repository \(freeze commitf0b5895\) before any confirmatory DPO run, but were not deposited with an external registry, so their timing rests on the repository history rather than on a third\-party timestamp\.

#### Data and split\.

The primary signals are MIT\-BIH Arrhythmia Database v1\.0\.0 \(DOI 10\.13026/C2F305\), with baseline\-wander, muscle\-artifact and electrode\-motion recordings from MIT\-BIH Noise Stress Test Database v1\.0\.0\. Records 201 and 202 are grouped as one subject\. Subjects are split before window construction into 27 train, 8 validation and 12 test subjects, yielding 1,000/300/500 fixed 30\-s requirements\. Beat annotations are used only for eligibility and the biomedical oracle and do not enter the prompt features\.

#### Pipeline DSL and target\.

Each generator emits a finite causal ECG pipeline with whitelisted high\-pass and low\-pass settings, filter family, notch setting, downsampling, derivative flag, integration window, adaptive threshold multiplier, refractory period and numeric format\. The deployment label is

F=valid/stable∧\[cycles≤Cr\]∧\[RAM≤Rr\]∧\[latency≤Lr\]\.F=\\text\{valid/stable\}\\wedge\[\\mathrm\{cycles\}\\leq C\_\{r\}\]\\wedge\[\\mathrm\{RAM\}\\leq R\_\{r\}\]\\wedge\[\\mathrm\{latency\}\\leq L\_\{r\}\]\.The biomedical label is

M=\[F1QRS≥0\.80\]∧\[MAER≤65ms\],M=\[F1\_\{\\rm QRS\}\\geq 0\.80\]\\wedge\[\\mathrm\{MAE\}\_\{R\}\\leq 65\\text\{ ms\}\],and the full target isy=F∧My=F\\wedge M\. Benchmark difficulty was resolved from 131,072 Sobol configurations before generator experiments:P⁡\(F\)=0\.5848P\(F\)=0\.5848,P⁡\(M∣F\)=0\.2408P\(M\\mid F\)=0\.2408, with semantic counts 54,421F=0F=0, 58,192F=1,M=0F=1,M=0, and 18,459F=1,M=1F=1,M=1\.

#### Fixed critic\.

One two\-head MLP critic is trained independently of the generator families and then frozen\. Its held\-out test AUC is 1\.000 for feasibility and 0\.930 for utility\. The same critic scores Llama, Gemma and Phi candidate pools\.

#### Models and generation\.

The frozen generator revisions are Llama\-3\.2\-3B\-Instruct \(revision0cb88a4\.\.\.\), Gemma\-3\-4B\-IT \(093f9f3\.\.\.\), and Phi\-4\-mini\-instruct \(cfbefac\.\.\.\)\. Each requirement hasK=16K=16samples at temperature 0\.8 and top\-p=0\.95p=0\.95\. The first prompt implementation was rejected before the preregistration freeze because it produced no usable Gemma outputs and almost no schema\-valid Llama outputs; no valid biomedical outcomes from that attempt were available for scientific tuning\. A shared prompt\-v2 was accepted by a parse\-only train pilot under a criterion fixed before inspecting those outputs, then hashed and frozen\. The Phi checkpoint required disabling incompatible remote model code before any model weights or completions were produced; all scientific settings remained unchanged\. These engineering deviations are retained in the package\.

#### Selection and pair construction\.

The scalar grid is\{\.001,\.003,\.01,\.03,\.1,\.3,1,3,10\}\\\{\.001,\.003,\.01,\.03,\.1,\.3,1,3,10\\\}\. Validation selection giveswsel=\.003,\.001,\.03w\_\{\\rm sel\}=\.003,\.001,\.03for Llama, Gemma and Phi\. A critic\-only normalized maximin of chosen/rejected feasibility and utility contrasts on the frozen train pools \(each contrast divided by its maximum over the grid; the gate\-free criterion of Appendix[O](https://arxiv.org/html/2610.29029#A15)\) giveswpair=1,\.1,\.3w\_\{\\rm pair\}=1,\.1,\.3\. Each rule constructs 4,000 deterministic top\-4\-versus\-bottom\-4 pairs\. Llama additionally includes a unit\-sum arm\. Exact pair overlap with the hard gate is 92\.3%/25\.6% for Llama inference/pair weights, 99\.5%/71\.1% for Gemma, and 79\.6%/31\.0% for Phi\. Llama pair\-weight and unit\-sum pairs are exactly identical\.

#### DPO and evaluation\.

All runs use sigmoid DPO withβ=\.1\\beta=\.1, two epochs, learning rate10−510^\{\-5\}, LoRA rank 16 \(α=32\\alpha=32, dropout \.05\), batch 2 with accumulation 8, and maximum length 768\. The matrix contains 12 Llama runs and 9 each for Gemma and Phi, three seeds per rule\. All 30 runs completed and all 30 aligned test pools contain 500 requirements by 16 candidates\. Evaluation uses one common hard\-feasibility\-then\-utility selector, not the training rule\. The primary endpoint is Answer@16; the preregistered gate\-equivalence margin is±\.04\\pm\.04\.

#### Per\-seed endpoints\.

Table[25](https://arxiv.org/html/2610.29029#A26.T25)lists all 30 runs\. No seed was removed or repeated\. Typical invalid outputs include numeric values in the notch field, quoted numeric notch values, unquotedQ15, and refractory values outside the finite schema\.

Table 25:All ECG DPO runs: Answer@16 \(parse rate\) per training seed under the common evaluator\. Llama pair\-weight and unit\-sum were trained on identical pair sets\.
#### Sensitivity to the number of test subjects \(post\-hoc\)\.

The frozen bootstrap resamples only 12 test subjects\. A leave\-one\-subject\-out jackknife witht11t\_\{11\}intervals gives\[−\.386,−\.273\]\[\-\.386,\-\.273\],\[\+\.043,\+\.194\]\[\+\.043,\+\.194\]and\[−\.264,−\.128\]\[\-\.264,\-\.128\]for pair minus inference \(Llama, Gemma, Phi\),\[−\.390,−\.283\]\[\-\.390,\-\.283\],\[\+\.043,\+\.194\]\[\+\.043,\+\.194\]and\[−\.253,−\.131\]\[\-\.253,\-\.131\]for pair minus gate, and\[−\.003,\+\.016\]\[\-\.003,\+\.016\],\[−\.002,\+\.001\]\[\-\.002,\+\.001\]and\[−\.018,\+\.010\]\[\-\.018,\+\.010\]for gate minus inference\. Dropping any single subject moves no estimate by more than0\.0240\.024\. Per\-seed contrasts are the informative caveat: pair minus gate is−\.21,−\.19,−\.18\-\.21,\-\.19,\-\.18for Phi, but−\.51,−\.52,\+\.02\-\.51,\-\.52,\+\.02for Llama and−\.01,\+\.12,\+\.24\-\.01,\+\.12,\+\.24for Gemma, so only the Phi sign is stable across training seeds\.

#### Output diversity and a requirement\-blind program \(post\-hoc\)\.

One pipeline \(IIR4 filter, 0\.5–25 Hz band, no notch, no downsampling, derivative on, 140 ms integration, threshold multiplier 0\.35, 220 ms refractory, Q15 arithmetic\) satisfies the full target on51\.2%51\.2\\%of test requirements regardless of the request\. It is already the most frequent Llama base output \(38\.5%38\.5\\%of train samples\)\. It accounts for7878–100%100\\%of parsed outputs in six of the eight Llama arms that parse, and for97\.197\.1–100%100\\%of parsed outputs in all three Phi pair\-weight seeds, versus00–8%8\\%in Phi gate and inference arms, which retain 79–113 distinct pipelines per 500×\\times16 pool\. Gemma never emits it\. Endpoints near0\.5120\.512therefore indicate a requirement\-blind policy, not partial success\.

#### Gate\-referenced versus gate\-free contrast criteria \(post\-hoc\)\.

Rebuilding the pairs of the frozen builder on the frozen train pools for every grid weight \(which reproduces the frozen gate pair sets exactly\), the critic contrasts at the gate coincide with those of the smallest weights: the gate’s feasibility contrast is matched within1\.2%1\.2\\%atw≤0\.01w\\leq 0\.01in every family, while its utility contrast equals the sweep maximum within0\.2%0\.2\\%\. The gate\-free maximin therefore moves away from the gate toward feasibility\. Selecting instead the grid weight nearest the gate in gate\-normalized critic\-contrast space givesw=0\.01w=0\.01\(Llama\),0\.0030\.003\(Gemma\) and0\.0030\.003\(Phi\), with exact pair overlap with the gate of87\.3%87\.3\\%,99\.0%99\.0\\%and94\.7%94\.7\\%\. Atwpairw\_\{\\rm pair\}, the oracle utility contrast falls from\.524\.524to\.461\.461\(Llama\) and from\.303\.303to\.259\.259\(Phi\) relative to the gate, while the oracle feasibility contrast is unchanged or slightly higher\. No policy was trained with the gate\-referenced weights; these numbers locate the criteria, they are not training outcomes\.

#### Interpretation\.

The study supports transfer of the audit and selection separations and the pair\-equivalence reading of the training interface\. It rejects the gate\-free contrast criterion as a transferable recipe; in its one informative family \(Phi\) the failure has the engine’s feasibility\-dominated signature and ends in a requirement\-blind program\. It does not establish that gate\-referenced contrast matching would succeed outside the near\-identical\-pair regime\.

## Appendix AAAudit, selection and reranker analyses: proofs and CPU\-only results

All analyses in this appendix are post hoc, CPU\-only, and use the frozen ECG pools, the frozen critic and the frozen common evaluator; scripts arev23\_\*\.pyandv24\_\*\.pyin the supplement\. No policy was trained for this appendix\.

### AA\.1Proof of Proposition[3](https://arxiv.org/html/2610.29029#Thmproposition3)

*\(i\), upper bound\.*LetN∼Bin⁡\(m,1−q\)N\\sim\\mathrm\{Bin\}\(m,1\-q\)be the number of revealed own failures andμ=m⁡\(1−q\)\\mu=m\(1\-q\)\. Conditional onN=kN=k, the revealedhhvalues are i\.i\.d\. in\[0,1\]\[0,1\]with meanAownA\_\{\\rm own\}, so by Hoeffding’s inequality\([Hoeffding, 1963](https://arxiv.org/html/2610.29029#bib.bib39)\)Pr⁡\[\|A^−Aown\|\>ε∣N=k\]≤2​e−2​k​ε2\\Pr\[\|\\hat\{A\}\-A\_\{\\rm own\}\|\>\\varepsilon\\mid N=k\]\\leq 2e^\{\-2k\\varepsilon^\{2\}\}\. The multiplicative Chernoff bound givesPr\[N<μ/2\]≤e−μ/8≤δ/2\\Pr\[N<\\mu/2\]\\leq e^\{\-\\mu/8\}\\leq\\delta/2whenμ≥8​log⁡\(2/δ\)\\mu\\geq 8\\log\(2/\\delta\), and onN≥μ/2N\\geq\\mu/2the conditional failure probability is at most2​e−μ​ε2≤δ/22e^\{\-\\mu\\varepsilon^\{2\}\}\\leq\\delta/2whenμ≥log⁡\(4/δ\)/ε2\\mu\\geq\\log\(4/\\delta\)/\\varepsilon^\{2\}\. A union bound completes the claim\.*\(i\), lower bound\.*Take two instances that share the positives,qq, and the inherited bin, but whose own bins haveAown=0A\_\{\\rm own\}=0andAown=1A\_\{\\rm own\}=1\. On the eventN=0N=0, which has probabilityqmq^\{m\}under both, the labelled sample has the same law, so any estimator has the same output distribution; since the targets differ by1\>2​ε1\>2\\varepsilon, it errs by more thanε\\varepsilonon at least one instance with probability at leastqm/2q^\{m\}/2\. Requiring this to be at mostδ\\deltagivesm≥log⁡\(1/\(2​δ\)\)/log⁡\(1/q\)≥q​log⁡\(1/\(2​δ\)\)/\(1−q\)m\\geq\\log\(1/\(2\\delta\)\)/\\log\(1/q\)\\geq q\\log\(1/\(2\\delta\)\)/\(1\-q\), usinglog⁡\(1/q\)≤\(1−q\)/q\\log\(1/q\)\\leq\(1\-q\)/q\.*\(i\),ε\\varepsilon\-dependent lower bound\.*Take two instances that share the positives,qqand the inherited bin, and whose own\-bin negatives all score either below every positive \(h=1h=1\) or above every positive \(h=0h=0\), withPr\[h=1\]=12\+32ε\\Pr\[h=1\]=\\tfrac\{1\}\{2\}\+\\tfrac\{3\}\{2\}\\varepsilonin one instance and12−32​ε\\tfrac\{1\}\{2\}\-\\tfrac\{3\}\{2\}\\varepsilonin the other\. Forε≤14\\varepsilon\\leq\\tfrac\{1\}\{4\}these are valid Bernoulli laws and the two values ofAownA\_\{\\rm own\}differ by3​ε\>2​ε3\\varepsilon\>2\\varepsilon\. A labelled negative reveals its bin, which has the same law under both, and, if it is an own failure, one Bernoulli draw\. Hence the KL divergence between the laws of anmm\-sample ism\(1−q\)KL\(Ber\(12\+32ε\)∥Ber\(12−32ε\)\)m\(1\-q\)\\,\\mathrm\{KL\}\(\\mathrm\{Ber\}\(\\tfrac\{1\}\{2\}\+\\tfrac\{3\}\{2\}\\varepsilon\)\\,\\\|\\,\\mathrm\{Ber\}\(\\tfrac\{1\}\{2\}\-\\tfrac\{3\}\{2\}\\varepsilon\)\)\. A direct one\-variable bound givesKL≤24​ε2\\mathrm\{KL\}\\leq 24\\varepsilon^\{2\}on0<ε≤140<\\varepsilon\\leq\\tfrac\{1\}\{4\}\(equivalently, withx=3​ε≤3/4x=3\\varepsilon\\leq 3/4,x​log⁡\(\(1\+x\)/\(1−x\)\)≤\(8/3\)​x2x\\log\(\(1\+x\)/\(1\-x\)\)\\leq\(8/3\)x^\{2\}\)\. Because the targets are more than2​ε2\\varepsilonapart, an estimate cannot be withinε\\varepsilonof both\. Bretagnolle–Huber\([Tsybakov, 2009](https://arxiv.org/html/2610.29029#bib.bib40)\)therefore implies that the larger of the two probabilities of error greater thanε\\varepsilonis at least14​exp⁡\(−24​m​\(1−q\)​ε2\)\\tfrac\{1\}\{4\}\\exp\(\-24m\(1\-q\)\\varepsilon^\{2\}\)\. Requiring it to be at mostδ<14\\delta<\\tfrac\{1\}\{4\}givesm⁡\(1−q\)≥log⁡\(1/\(4​δ\)\)/\(24​ε2\)m\(1\-q\)\\geq\\log\(1/\(4\\delta\)\)/\(24\\varepsilon^\{2\}\)\. Together with the upper bound, the passive budget isΘ⁡\(log⁡\(1/δ\)/\(ε2​\(1−q\)\)\)\\Theta\(\\log\(1/\\delta\)/\(\\varepsilon^\{2\}\(1\-q\)\)\)up to universal constants\.*\(ii\)\.*FromAi=\(1−qi\)​Aown\+qi​AcrossA\_\{i\}=\(1\-q\_\{i\}\)A\_\{\\rm own\}\+q\_\{i\}A\_\{\\rm cross\}fori=1,2i=1,2,Aown=\(q2​A1−q1​A2\)/\(q2−q1\)A\_\{\\rm own\}=\(q\_\{2\}A\_\{1\}\-q\_\{1\}A\_\{2\}\)/\(q\_\{2\}\-q\_\{1\}\); substituting estimates with errorseie\_\{i\},\|ei\|≤εi\|e\_\{i\}\|\\leq\\varepsilon\_\{i\}, changes the right\-hand side by\(q2​e1−q1​e2\)/\(q2−q1\)\(q\_\{2\}e\_\{1\}\-q\_\{1\}e\_\{2\}\)/\(q\_\{2\}\-q\_\{1\}\)\. ∎

### AA\.2Proof of Proposition[6](https://arxiv.org/html/2610.29029#Thmproposition6)

With continuous coordinates there are no ties almost surely\. Order candidates by decreasingsFs\_\{F\}\. Candidateiiin this order is dominated iff some earlier candidate has largersMs\_\{M\}, so it is Pareto iff itssMs\_\{M\}is the maximum of the firstiivalues\. By independence ofsFs\_\{F\}andsMs\_\{M\}, thesMs\_\{M\}sequence in this order is i\.i\.d\. continuous, and its record indicators are independent Bernoulli\(1/i\)\(1/i\)\(Rényi’s record theorem;[Bentley et al\. 1978](https://arxiv.org/html/2610.29029#bib.bib35)\)\. Summing gives𝔼​\|Par\|=HK\\mathbb\{E\}\|\\operatorname\{Par\}\|=H\_\{K\};\|Par\|=1\|\\operatorname\{Par\}\|=1iff the first candidate holds the overall maximum, probability1/K1/K\. Under comonotone heads the top\-sFs\_\{F\}candidate also has topsMs\_\{M\}and dominates all others\. A Monte\-Carlo check with20 00020\\,000pools perK∈\{2,4,8,16,64\}K\\in\\\{2,4,8,16,64\\\}matchesHKH\_\{K\}and1/K1/Kto within0\.0060\.006and0\.0040\.004\. ∎

### AA\.3Proof of Proposition[7](https://arxiv.org/html/2610.29029#Thmproposition7)\(iv\)

Letδt=‖θtP−θtQ‖\\delta\_\{t\}=\\\|\\theta^\{P\}\_\{t\}\-\\theta^\{Q\}\_\{t\}\\\|andΔ=supθ‖∇LP​\(θ\)−∇LQ​\(θ\)‖≤2​G​dTV​\(P,Q\)\\Delta=\\sup\_\{\\theta\}\\\|\\nabla L\_\{P\}\(\\theta\)\-\\nabla L\_\{Q\}\(\\theta\)\\\|\\leq 2G\\,d\_\{\\rm TV\}\(P,Q\)by part \(iii\)\. Thenδt\+1≤∥\(θtP−η∇LP\(θtP\)\)−\(θtQ−η∇LP\(θtQ\)\)∥\+ηΔ\\delta\_\{t\+1\}\\leq\\\|\(\\theta^\{P\}\_\{t\}\-\\eta\\nabla L\_\{P\}\(\\theta^\{P\}\_\{t\}\)\)\-\(\\theta^\{Q\}\_\{t\}\-\\eta\\nabla L\_\{P\}\(\\theta^\{Q\}\_\{t\}\)\)\\\|\+\\eta\\Delta\. For convexLL\-smoothLPL\_\{P\}andη≤2/L\\eta\\leq 2/Lthe gradient step is non\-expansive\([Hardt et al\., 2016](https://arxiv.org/html/2610.29029#bib.bib38)\), soδt\+1≤δt\+η​Δ\\delta\_\{t\+1\}\\leq\\delta\_\{t\}\+\\eta\\DeltaandδT≤η​T​Δ\\delta\_\{T\}\\leq\\eta T\\Delta\. With smoothness alone the step is\(1\+η​L\)\(1\+\\eta L\)\-Lipschitz, givingδT≤Δ​∑t<Tη​\(1\+η​L\)t=Δ⁡\(\(1\+η​L\)T−1\)/L\\delta\_\{T\}\\leq\\Delta\\sum\_\{t<T\}\\eta\(1\+\\eta L\)^\{t\}=\\Delta\(\(1\+\\eta L\)^\{T\}\-1\)/L\. Any evaluation functional that isΛ\\Lambda\-Lipschitz inθ\\thetathen differs by at mostΛ​δT\\Lambda\\delta\_\{T\}\. The DPO objective is convex for log\-linear policies; for LoRA fine\-tuning neither assumption is guaranteed, which is why the main text treats overlap as a predictor, not a guarantee, for endpoints\. ∎

### AA\.4Selection ambiguity versus pool size

For each requirement we drawKKof the1616stored candidates \(2020draws forK<16K<16\), keep parsed candidates, collapse identical score pairs, and compute the Pareto set\.HnH\_\{n\}and1/n1/nare the independence values of Proposition[6](https://arxiv.org/html/2610.29029#Thmproposition6)at the observed numbernnof distinct points; the permutation null shufflessMs\_\{M\}within the pool\. The range is the per\-pool outer range of Answer@KKover audit\-equivalent composites, averaged over requirements\.

Table 26:Selection ambiguity on ECG test pools as a function ofKK\(post hoc\)\.The permutation null reproducesHnH\_\{n\}within0\.050\.05in every row, confirming the proposition on real score marginals; the observed Pareto sets are smaller becausesFs\_\{F\}andsMs\_\{M\}are positively dependent within pools, and they still grow roughly logarithmically inKK\.

### AA\.5Audit decomposition on post\-DPO pools

Table[27](https://arxiv.org/html/2610.29029#A27.T27)lists every trained ECG arm with at least one evaluable pool \(a pool is evaluable when its parsed candidates contain both oracle classes\)\. “Own support” is the share of evaluable pools whose own\-failure bin \(F=0F=0\) is non\-empty; “erased” is the share withq≥max⁡\(A,1−A\)q\\geq\\max\(A,1\-A\), which includes every pool withq=1q=1\.

Table 27:Deployability\-head audit on base and post\-DPO ECG test pools \(post hoc\)\.*Designed re\-injection\.*For the1212Llama and Phi gate/inference runs we draw200200positives from the trained pool and400400negatives mixing the trained pool’s inherited bin with base\-policy own failures of the same family atq1=0\.2q\_\{1\}=0\.2andq2=0\.8q\_\{2\}=0\.8, and invert the2×22\\times 2system \(200200repetitions\)\. Against the direct pooled valueAown=1\.000A\_\{\\rm own\}=1\.000the estimates average 0\.998–1\.003 with RMSE 0\.016–0\.020\. Because the deployability head is near\-perfect, this checks the procedure rather than detecting degradation\.

### AA\.6Label budget as a function ofqq

Utility head, pooled base test candidates; positivesy=1y=1, own binM=0M=0, inherited binF=0∧M=1F=0\\wedge M=1\. Each of2 0002\\,000repetitions labelsmmnegatives from a mixture with weightqqand averageshhover the revealed own failures\.

Table 28:Passive audit cost \(post hoc\)\. The scaled errorm⁡\(1−q\)⋅\\sqrt\{m\(1\-q\)\}\\cdotRMSE is constant, as Proposition[3](https://arxiv.org/html/2610.29029#Thmproposition3)predicts, until own failures stop appearing\.
### AA\.7Seed\-and\-subject intervals for the ECG contrasts

The frozen intervals of Table[40](https://arxiv.org/html/2610.29029#A32.T40)condition on the three training seeds\. Table[29](https://arxiv.org/html/2610.29029#A27.T29)resamples, in each of5 0005\\,000replicates, the three seeds of each rule independently and the1212test subjects, and recomputes the mean Answer@16 difference under the frozen common evaluator\.

Table 29:Two\-level \(seed and subject\) bootstrap intervals \(post hoc\)\.
### AA\.8Prediction\-registered convex DPO reranker

*Design, frozen before running\.*The design and four predictions were written tov23b\_PREDICTIONS\.md\(SHA\-256d6863c3a8fb1ff40…\) before any training run; this study is post hoc relative to the ECG preregistration\. For Llama and Phi \(Gemma excluded a priori for lack of supply\) a rerankerπθ​\(i∣r\)∝exp⁡\(θ⊤​ϕ​\(r,i\)\)\\pi\_\{\\theta\}\(i\\mid r\)\\propto\\exp\(\\theta^\{\\top\}\\phi\(r,i\)\)over the parsed candidates of each frozen base pool uses DSL one\-hot features and their products with standardized requirement features \(588588dimensions\); the reference is uniform \(θ0=0\\theta\_\{0\}=0\), so the DPO log\-ratio of a same\-pool pair isθ⊤​\(ϕc−ϕr\)\\theta^\{\\top\}\(\\phi\_\{c\}\-\\phi\_\{r\}\)and the objective is convex\. Sigmoid DPO withβ=0\.1\\beta=0\.1, minibatch1616, two epochs over the rule’s4 0004\\,000pairs, ten shuffle seeds; the learning rate was chosen once per family from\{1,3,10,30\}\\\{1,3,10,30\\\}by the gate\-trained reranker’s validation success \(seed 0\) and frozen \(3030in both, the grid edge\)\. Pairs are rebuilt with the frozen builder and reproduce the frozen gate,wselw\_\{\\rm sel\}andwpairw\_\{\\rm pair\}files exactly\. The endpoint is expected per\-draw target success on the test pools; intervals resample seeds and the1212test subjects\. Uniform reranking scores0\.3600\.360\(Llama\) and0\.4430\.443\(Phi\)\.

*Predictions and outcomes\.*P1 \(gate\-referenced weight within±0\.04\\pm 0\.04of the gate in both families\): held\. P2 \(Phi’s gate\-freewpairw\_\{\\rm pair\}not within±0\.04\\pm 0\.04\): held, with a small effect \(−0\.016\-0\.016\)\. P3 \(\|\|gap\|\|negatively rank\-correlated with overlap\): held, Spearman−0\.60\-0\.60\(Llama\) and−0\.93\-0\.93\(Phi\)\. P4 \(full\-batch GD distances respect Proposition[7](https://arxiv.org/html/2610.29029#Thmproposition7)\(iv\)\): held; distances are monotone in1−1\-overlap \(Spearman1\.01\.0\) and at most1\.1%1\.1\\%of the bound, which is therefore valid but loose\.*Post hoc, after seeing results:*the test top\-1 choice is the requirement\-blind program for25%25\\%\(gate\),27%27\\%\(gate\-referenced\) and67%67\\%\(wpairw\_\{\\rm pair\}\) of Phi requirements, and48%48\\%,52%52\\%and86%86\\%for Llama\.

Table 30:Convex DPO reranker on frozen ECG candidate pools, ten seeds per rule \(prediction\-registered, CPU\-only\)\. “dist” is the full\-batch GD parameter distance to the gate after200200steps\.

## Appendix ABRicher audit information, realistic selection, and rule\-level pair geometry

All results in this appendix are post hoc and CPU\-only on frozen artifacts \(v24\_trimming\_bounds\.py,v24\_realistic\_selector\.py,v24\_rule\_geometry\.py\); no policy was trained\.

### AB\.1Identified set when every negative’s score is observed

Fix a pool withnnnegatives, of whichnown=\(1−q\)​nn\_\{\\rm own\}=\(1\-q\)nfailFF, and leth\(1\)≤⋯≤h\(n\)h\_\{\(1\)\}\\leq\\cdots\\leq h\_\{\(n\)\}be the sorted valuesh⁡\(x\)h\(x\)over negatives \(the share of the pool’s positives ranked abovexx, ties12\\tfrac\{1\}\{2\}\)\. Suppose the auditor observes all scores and all outcome labelsyy, andqq, but not which negatives failFF\. Then

Aown∈\[1nown​∑i≤nownh\(i\),1nown​∑i\>n−nownh\(i\)\],A\_\{\\rm own\}\\in\\Big\[\\tfrac\{1\}\{n\_\{\\rm own\}\}\\textstyle\\sum\_\{i\\leq n\_\{\\rm own\}\}h\_\{\(i\)\},\\;\\tfrac\{1\}\{n\_\{\\rm own\}\}\\sum\_\{i\>n\-n\_\{\\rm own\}\}h\_\{\(i\)\}\\Big\],and the interval is sharp\.*Proof\.*AownA\_\{\\rm own\}is the mean ofhhover the own bin, a subset of sizenownn\_\{\\rm own\}of the negatives; every such subset is compatible with the observations because bin membership is unobserved andqqfixes only its size, and the extreme means are attained by the lowest and highestnownn\_\{\\rm own\}values\. In the population version this is the trimming bound of[Horowitz and Manski \(1995\)](https://arxiv.org/html/2610.29029#bib.bib15)\. It equals\[0,1\]\[0,1\]iff at leastnownn\_\{\\rm own\}negatives haveh=0h=0and at leastnownn\_\{\\rm own\}haveh=1h=1\. Ifqqis known only to an interval, the identified set is the union over the admissiblenownn\_\{\\rm own\}\. ∎

Table[31](https://arxiv.org/html/2610.29029#A28.T31)compares three information structures on the ECG deployability head: \(I0\)\(A,q\)\(A,q\)only, Eq\.[4](https://arxiv.org/html/2610.29029#S4.E4); \(I1\) plus every negative’s score; \(I2\) as I1 but withqqreplaced by a95%95\\%Clopper–Pearson interval from5050labelled negatives\. Every interval contains the directly measuredAownA\_\{\\rm own\}where the own bin has support \(I2:≥98%\\geq 98\\%of pools\)\. Observing the scores barely helps, because withK=16K=16most negatives are ranked below all positives or above them\.

Table 31:Mean interval width / share of evaluable pools with an uninformative interval \(width≥0\.999\\geq 0\.999\) under three information structures\. Post\-DPO Llama pools have no own\-bin support; pair\-weight arms have almost no evaluable pools \(Phi:00,22,2929\)\.
### AB\.2Selection with a validation\-tuned common weight

Proposition[5](https://arxiv.org/html/2610.29029#Thmproposition5)bounds what an audit leaves open\. A deployed selector uses one common weightwwinw​sF\+sMw\\,s\_\{F\}\+s\_\{M\}chosen on validation Answer@16\. Table[32](https://arxiv.org/html/2610.29029#A28.T32)reports, on the base test pools, the test value at the frozenwselw\_\{\\rm sel\}and the best and worst values over the frozen grid; all grid points share every component AUC\.

Table 32:Validation\-tuned common weight versus the test range over audit\-equivalent common weights \(Answer@16, ECG base test pools\)\.
### AB\.3Rule\-level pair geometry for all trained scalar rules

Table[33](https://arxiv.org/html/2610.29029#A28.T33)lists every trained scalar rule with its exact pair overlap with the gate, the bottleneck oracle contrastb⁡\(r\)=min⁡\{Δ​Fr/Δ​Fgate,Δ​Mr/Δ​Mgate\}b\(r\)=\\min\\\{\\Delta F\_\{r\}/\\Delta F\_\{\\rm gate\},\\Delta M\_\{r\}/\\Delta M\_\{\\rm gate\}\\\}of Appendix[Q](https://arxiv.org/html/2610.29029#A17), and its endpoint\. A family is*informative*for training iff its hard\-gate arm exceeds its reference by more than the±0\.04\\pm 0\.04margin \(ECG: the single requirement\-blind pipeline,0\.5120\.512; engine: the base policy’s supply@16,0\.2850\.285, an upper bound on its Answer@16\); the rule does not use the gaps\. It was fixed after the endpoints were known and is not part of the ECG preregistration\. Without it, all three rules with overlap≥0\.79\\geq 0\.79match the gate within0\.0070\.007\(Llama and Gemma inference weights against degenerate gates\), and the next\-highest overlap, Gemma’s pair weight at0\.710\.71, does not \(\+0\.119\+0\.119, with every arm far below the requirement\-blind program\)\. Among the seven informative rules, the one with overlap≥0\.79\\geq 0\.79matches the gate; five of six below miss by≥0\.08\\geq 0\.08\. All four withb<0\.5b<0\.5miss, but among low\-overlap rules withb≥0\.5b\\geq 0\.5one matches \(enginew=0\.1w=0\.1\) and one misses \(Phi pair, whose policies collapse onto the requirement\-blind program\)\. Over the seven informative rules the Spearman correlation of the signed gap is0\.540\.54withbband−0\.21\-0\.21with overlap\. Neither statistic is sufficient in the low\-overlap regime; overlap near one is sufficient by Proposition[7](https://arxiv.org/html/2610.29029#Thmproposition7)\.

Table 33:All eleven trained scalar rules \(Answer@16 under the common evaluator; three seeds\)\. Engine values from Table[39](https://arxiv.org/html/2610.29029#A32.T39)and Appendix[Q](https://arxiv.org/html/2610.29029#A17); ECG from Table[40](https://arxiv.org/html/2610.29029#A32.T40)and the frozen pair sets\.

## Appendix ACAUC\-change decomposition and engine selection controls

#### Mixture\-shift versus ordering\-drift decomposition\.

Across two policies, the AUC change can also be decomposed into mixture shift and conditional\-ordering drift:

At−As=\\displaystyle A\_\{t\}\-A\_\{s\}=\{\}\(qt−qs\)​\(Acrosss−Aowns\)\\displaystyle\(q\_\{t\}\-q\_\{s\}\)\(A\_\{\\rm cross\}^\{s\}\-A\_\{\\rm own\}^\{s\}\)\(7\)\+\(1−qt\)​\(Aownt−Aowns\)\+qt​\(Acrosst−Acrosss\)\.\\displaystyle\+\(1\-q\_\{t\}\)\(A\_\{\\rm own\}^\{t\}\-A\_\{\\rm own\}^\{s\}\)\+q\_\{t\}\(A\_\{\\rm cross\}^\{t\}\-A\_\{\\rm cross\}^\{s\}\)\.The first term changes only composition; the last two change conditional ordering\. Both mechanisms can therefore coexist\. The identity extends to any disjoint failure\-bin partition and requires no independence assumptions\.

#### Rules that reduce the saturated component’s influence recover selection\.

Holding the candidate pools and critic fixed, the raw unit sum reachesη=0\.10±0\.02\\eta=0\.10\\pm 0\.02on matched 1\.5B pools, rank averaging0\.58±0\.010\.58\\pm 0\.01, a validation\-tuned feasibility weight of0\.0050\.005reaches0\.88±0\.020\.88\\pm 0\.02, the soft log\-probability rule reaches0\.89±0\.000\.89\\pm 0\.00, and threshold\-then\-mass reaches0\.92±0\.010\.92\\pm 0\.01\. At 7B, the raw sum remains poor \(0\.05±0\.020\.05\\pm 0\.02\) while threshold\-then\-mass reaches0\.94±0\.040\.94\\pm 0\.04\(full sweep in Appendix[M](https://arxiv.org/html/2610.29029#A13)\)\.

The residual feasibility ordering is not useless \(permuting it within the passing set lowers raw\-sum answer on gated 7B pools from0\.2370\.237to0\.2090\.209, post\-hoc\); the problem is excessive influence, not absence of cross\-signal\.

#### Controls\.

A holistic on\-target classifier \(held\-out AUC0\.9740\.974\) avoids the component\-AUC cliff but captures onlyη=0\.49\\eta=0\.49–0\.600\.60, versus0\.900\.90–0\.940\.94for the gated two\-head critic\. This is not evidence that decomposition is universally superior: the mass regressor contains ordinal distance\-to\-budget information\. GSM8K shows the opposite saturation geometry\. Its format\-head within\-pool SD falls from8\.68\.6to0\.020\.02, so the tested composites effectively reduce to the correctness head and agree within0\.010\.01\(Appendix[C](https://arxiv.org/html/2610.29029#A3)\)\.

## Appendix ADAudit inversion and the refresh study: proofs, protocol and full results

### AD\.1Proof of Proposition[2](https://arxiv.org/html/2610.29029#Thmproposition2)

*\(i\)*Subtract Eq\.[3](https://arxiv.org/html/2610.29029#S4.E3)forffandggin the same pool \(sameqq\); the sign of\(1−q\)​\(−Δown\)\+q​Δcross\(1\-q\)\(\-\\Delta\_\{\\rm own\}\)\+q\\Delta\_\{\\rm cross\}is positive iffq\>Δown/\(Δown\+Δcross\)q\>\\Delta\_\{\\rm own\}/\(\\Delta\_\{\\rm own\}\+\\Delta\_\{\\rm cross\}\)\.*\(ii\)*In a pool with candidate lawPP,AUC\(s\)=Pr\[s\(X\)\>s\(X′\)\]\+12Pr\[s\(X\)=s\(X′\)\]\\operatorname\{AUC\}\(s\)=\\Pr\[s\(X\)\>s\(X^\{\\prime\}\)\]\+\\tfrac\{1\}\{2\}\\Pr\[s\(X\)=s\(X^\{\\prime\}\)\]forX∼P\(⋅∣y=1\)X\\sim P\(\\cdot\\mid y=1\),X′∼P\(⋅∣y=0\)X^\{\\prime\}\\sim P\(\\cdot\\mid y=0\)independent\. The likelihood ratiodP\(⋅∣y=1\)/dP\(⋅∣y=0\)dP\(\\cdot\\mid y=1\)/dP\(\\cdot\\mid y=0\)is a strictly increasing function ofηy\\eta\_\{y\}, and any scorer that is a strictly increasing transform of it maximizes this probability\([Clémençon et al\., 2008](https://arxiv.org/html/2610.29029#bib.bib13), Proposition 1\)\.*\(iii\)*Sincey=F∧My=F\\wedge M,y=1y=1impliesF=ϕ⁡\(x\)=1F=\\phi\(x\)=1, soηy​\(x\)=0\\eta\_\{y\}\(x\)=0wheneverϕ⁡\(x\)=0\\phi\(x\)=0, whileηy​\(x\)\>0\\eta\_\{y\}\(x\)\>0almost surely underP\(⋅∣y=1\)P\(\\cdot\\mid y=1\); hence every positive outranks every own failure andaown​\(ηy\)=1a^\{\\rm own\}\(\\eta\_\{y\}\)=1\. On\{ϕ=1\}\\\{\\phi=1\\\}we havey=My=Mandηy=Pr⁡\[M=1∣x\]\\eta\_\{y\}=\\Pr\[M=1\\mid x\], which is the optimal ranker for the cross subproblem by \(ii\) applied to that subpopulation, soacross​\(ηy\)≥12a^\{\\rm cross\}\(\\eta\_\{y\}\)\\geq\\tfrac\{1\}\{2\}\(the value of any constant scorer\), with equality iffηy\\eta\_\{y\}is a\.s\. constant there\. The oracleϕ\\phihasaown=1a^\{\\rm own\}=1andacross=12a^\{\\rm cross\}=\\tfrac\{1\}\{2\}; substituting in Eq\.[3](https://arxiv.org/html/2610.29029#S4.E3)givesA⁡\(ηy\)−A⁡\(ϕ\)=q⁡\(across​\(ηy\)−12\)A\(\\eta\_\{y\}\)\-A\(\\phi\)=q\(a^\{\\rm cross\}\(\\eta\_\{y\}\)\-\\tfrac\{1\}\{2\}\)\.□\\square

*Remark \(support\)\.*Refresh data drawn from a policy whose negatives contain no own failures carry no information about how a head orders own failures: any two heads that agree on the refresh support fit it equally well, however they rank off\-supportFF\-failures\. This is Proposition[1](https://arxiv.org/html/2610.29029#Thmproposition1)withπown=0\\pi\_\{\\rm own\}=0, now read as a statement about learning rather than auditing\.

### AD\.2Protocol

Data are the frozen ECG pools of Appendix[Z](https://arxiv.org/html/2610.29029#A26): base\-policy train pools \(Llama, Phi\), base\-policy test pools \(Llama, Phi, and Gemma as an unseen generator\), and the test pools of the six post\-DPO policies per family trained with gate or inference\-weight pairs\. Features are the frozen critic’s5555inputs \(requirement descriptors and one\-hot DSL fields\)\. Head classes: an MLP with the critic’s architecture \(256256\-128128\-6464, Adam, batch256256, up to4040epochs\) and histogram gradient boosting \(200200trees, no early stopping\)\.*Label\-mixture heads*\(P1\) are trained from scratch on the base train pool with each candidate duplicated and labelledyywith probabilityλ\\lambdaandFFotherwise, for three seeds\.*Refresh heads*continue training the specialistSS\(λ=0\\lambda=0\): the MLP by2020epochs ofpartial\_fit, the GBM by100100additional trees on the refresh data\. Refresh data are all parsed candidates of the post\-DPO pools whose requirement belongs to a random half of the test subjects \(seeds100100–109109\);RmixR\_\{\\rm mix\}adds an equal number of base\-train candidates drawn uniformly\. All metrics use the complementary subject half\. The monitor is the within\-pool full\-target AUC of Section[2](https://arxiv.org/html/2610.29029#S2), averaged over the six post\-DPO policies; canonicalFF\-AUC is computed over all parsed candidates of the base pools; the gate selects the highest frozen utility score among candidates with head probability≥0\.5\\geq 0\.5and otherwise the head’s top candidate\.

### AD\.3Where the interface ingredients already appear

The exact controller we analyze is a composition of interfaces, not a claim about one named product\. Table[34](https://arxiv.org/html/2610.29029#A30.T34)records the closest modern ingredients\. Rubric systems expose criterion\-level judgments but ultimately aggregate them for policy optimization; policy\-aware rubric work explicitly measures saturation; and separate reward\-model work refreshes models from fresh policy\-interaction data\. At LLM scale, POW3R reports1717–26%26\\%saturated and2020–33%33\\%dead rubric criteria across its fresh\-policy diagnostics, so criterion informativeness is visibly policy\-dependent even before any maintenance decision is made\([Tyagi et al\., 2026](https://arxiv.org/html/2610.29029#bib.bib43)\)\. The unsafe loop arises when an aggregate outcome statistic is reused as the maintenance signal for one component head\.

Table 34:Contemporary systems instantiate the ingredients of the analyzed interface\. We do not claim that any cited system implements the exact audit\-trigger\-refresh controller end to end\.
### AD\.4Registered predictions and outcomes

The predictions file was hashed \(SHA\-256ac50e1a1…\) before the confirmatory run and is included in the supplement with the exploratory pilot, which used refitting from scratch rather than continued training and one split seed\. P1 \(pure component head ranked last on the post\-DPO audit, all four settings\):held\. P2 \(Phi monitor repaired by≥0\.05\\geq 0\.05in both classes\):failed\(\+0\.042\+0\.042,−0\.012\-0\.012\)\. P3 \(outcome refresh lowers off\-policyFF\-AUC by≥0\.05\\geq 0\.05, all four\):held\. P4 \(re\-injection within0\.010\.01ofSS, all four\):held\. P5 \(P3 and P4 on the unseen generator\):held\. P6 \(more gate violations underRyR\_\{y\}thanSS, MLP, both families\):held\.

Table 35:P1: on\-policy audit AUC of label\-mixture heads\(1−λ\)​F\+λ​y\(1\-\\lambda\)F\+\\lambda y\(mean of three seeds\)\. The pure component head \(λ=0\\lambda=0\) is lowest in every row labelled “audit”\.Table 36:Refresh minus deployed specialist, paired by subject split \(mean; in parentheses the number of the1010splits with a positive difference\)\.
### AD\.5Closed\-loop trigger analysis

This post\-hoc analysis composes already frozen quantities; it trains no new head\. For each of the four family×\\timeshead\-class settings, let the controller compare the pure specialist’s base\-policy audit to its post\-DPO audit \(Table[35](https://arxiv.org/html/2610.29029#A30.T35)\) and chooseRyR\_\{y\}iff the drop is at leastτ\\tau\. If it does not trigger, it leaves the deployed specialistSSunchanged\. Table[37](https://arxiv.org/html/2610.29029#A30.T37)reports the resulting mean over the four primary settings\. A reference\-distribution component controller using the same threshold on off\-policyFF\-AUC triggers no refresh becauseSSretainsFF\-AUC1\.0001\.000in all four settings\. The threshold sweep is illustrative rather than preregistered; its purpose is to test whether the measurement error can actually induce the harmful action posited by the control\-loop framing\.

Table 37:Counterfactual controller on frozen ECG heads\. “Triggered” is the number of the four primary family×\\timesclass settings in which aggregate audit AUC falls by at leastτ\\taufrom base to post\-DPO\. Triggered settings deployRyR\_\{y\}; others keepSS\. No\-refresh/component\-specific control hasFF\-AUC1\.0001\.000, violation\.148\.148, Answer@16\.530\.530\.Figure 12:Frozen closed\-loop threshold sweep\.Left: aggregate\-audit triggers deploy the already\-trained outcome\-refresh head and reduce canonicalFF\-AUC; the dotted secondary axis is the number of the four primary settings that trigger\. Right: the same controller raises constraint violations and lowers Answer@16\. Dashed horizontal lines are the no\-refresh/reference\-FFcontroller\.
### AD\.6Natural multi\-label semantic stress test: GoEmotions

We use the official GoEmotions development and test splits\([Demszky et al\., 2020](https://arxiv.org/html/2610.29029#bib.bib41)\), with5,4265\{,\}426and5,4275\{,\}427Reddit comments\. Each comment can carry multiple human\-annotated emotion labels\. The componentFFis fixed throughout as absence of*annoyance*; the development/test sets contain303/320303/320annoyance\-labelled own failures\. We train Laplace\-smoothed multinomial Naive Bayes heads on the development split over a fixed bag\-of\-words vocabulary \(terms occurring at least twice; at most10,00010\{,\}000\)\. This small transparent model is intentional: the experiment tests the audit geometry without conflating it with a large verifier’s capacity\.

#### Near\-boundary control fixed in v31\.

WithMM= absence of*disapproval*, the test set hasq=\.434q=\.434\. TheFFspecialist has\(Aown,Across,AF\)=\(\.683,\.569,\.679\)\(A\_\{\\rm own\},A\_\{\\rm cross\},A\_\{F\}\)=\(\.683,\.569,\.679\); the deliberately entangledsleak=z⁡\(sF\)\+z⁡\(sM\)s\_\{\\rm leak\}=z\(s\_\{F\}\)\+z\(s\_\{M\}\)has\(\.646,\.631,\.640\)\(\.646,\.631,\.640\)and predictedq∗=\.374q^\{\*\}=\.374\. At the natural mixture the audit difference is only\.006\.006and its 500\-resample bootstrap interval\[−\.009,\.019\]\[\-\.009,\.019\]spans zero\. This is therefore a useful near\-boundary sanity check rather than a claim of naturally observed inversion\.

#### Natural high\-qqpositive control\.

After the near\-boundary result, an explicitly post\-hoc exploratory scan kept the sameFFspecialist and varied the second annotated criterion; the scan identified absence of*gratitude*as a high\-support natural positive control\. We then reran this pair under the original frozen vocabulary/protocol\. The test set contains4,7574\{,\}757target positives,320320own failures and350350cross failures, henceq=\.522q=\.522\. The unchangedFFspecialist hasAown=\.665A\_\{\\rm own\}=\.665,Across=\.238A\_\{\\rm cross\}=\.238, canonicalFF\-AUC\.679\.679, and aggregate outcome AUC\.442\.442\. Withsλ=z⁡\(sF\)\+λ​z​\(sM\)s\_\{\\lambda\}=z\(s\_\{F\}\)\+\\lambda z\(s\_\{M\}\)atλ=1\\lambda=1, canonicalFF\-AUC falls to\.570\.570andAownA\_\{\\rm own\}to\.589\.589, whileAcrossA\_\{\\rm cross\}rises to\.765\.765\. Proposition[2](https://arxiv.org/html/2610.29029#Thmproposition2)predictsq∗=\.126q^\{\*\}=\.126\(bootstrap\[\.092,\.153\]\[\.092,\.153\]\), so the naturally observed mixture lies well past the crossing: the aggregate audit prefers the worse component verifier,\.681\.681versus\.442\.442\. A 500\-resample bootstrap interval for the audit difference is\[\.210,\.270\]\[\.210,\.270\], and the canonicalFF\-AUC gap favoring the specialist is\[\.089,\.128\]\[\.089,\.128\]\. Because the second criterion was selected post hoc, these intervals quantify stability of this fixed official test split; they are not a confirmatory multiplicity\-adjusted discovery claim\.

Table 38:Leakage\-strength sweep for the natural annoyance/gratitude control\. The specialist is theλ=0\\lambda=0reference: canonicalFF\-AUC\.679\.679and aggregate audit\.442\.442at observedq=\.522q=\.522\. Increasing cross\-component leakage makes the component classifier worse while making the aggregate audit better\.
Figure 13:Natural semantic inversion and a severity sweep\.Left: for the post\-hoc annoyance/gratitude positive control, the observedq=\.522q=\.522lies well past the analytic crossing\. Right: increasing cross\-component leakage monotonically degrades canonicalFF\-AUC while the outcome audit improves over most of the sweep\.
#### Support depletion and targeted replay\.

We next retrain the correctFFclassifier while retaining allF=1F=1development examples and a nested number of annoyance\-labelled failures\. CanonicalFF\-AUC rises from\.537\.537with two own failures to\.545\.545\(8\),\.594\.594\(64\),\.638\.638\(128\),\.671\.671\(256\), and\.679\.679with all303303failures\. The original fixed 8\+80 own\-failure replay arm reaches\.615\.615\. To distinguish targeted failure support from generic extra data, we additionally repeat 100 deterministic draws\. Adding 80 random development examples to the 8\-failure arm adds only4\.42±1\.964\.42\\pm 1\.96own failures on average and raises canonicalFF\-AUC only to\.550±\.004\.550\\pm\.004\. Adding 80 targeted own failures instead reaches\.606±\.008\.606\\pm\.008\. At the development\-set prevalence, a uniform sample would require about1,4331\{,\}433examples in expectation to contain 80 own failures\. The rescue is therefore about reconstructing the missing semantic support, not merely increasing the update set\.

Figure 14:Failure\-targeted replay is sample\-efficient\.Starting from only eight own failures, 80 indiscriminate examples add few relevant failures and barely move canonicalFF\-AUC; 80 explicit own failures recover substantially more component discrimination\. Error bars show SD over 100 deterministic replay draws; the full\-support bar uses all 303 development failures\.The exact script, source blob SHAs and frozen v31/v32 JSON summaries are included in the supplement\. All GoEmotions analyses are post hoc; the natural\-positive\-control scan and replay comparison were added after v31 and are labelled as such\.

### AD\.7Additional constructed semantic\-language control: AG News

This post\-hoc stress test uses the 2,000\-row AG News sample distributed in the OpenAI Cookbook \(blob SHAf63d8be6\.\.\.\) and the standard four topic labels of[Zhang et al\. \(2015\)](https://arxiv.org/html/2610.29029#bib.bib14)\. We concatenate title and description and defineF=𝟏\{topic≠World\}F=\\mathbf\{1\}\\\{\\text\{topic\}\\neq\\text\{World\}\\\},M=𝟏\{topic≠Sports\}M=\\mathbf\{1\}\\\{\\text\{topic\}\\neq\\text\{Sports\}\\\}, soy=F∧My=F\\wedge Miff the topic is Business or Sci/Tech\. Thus World examples are the own\-failure bin and Sports examples are the cross\-failure bin\. The construction is deliberately simple: it supplies semantic natural\-language components with ground\-truth labels while retaining the exact two\-bin geometry used by the theory\.

For each of 24 deterministic 65/35 hash splits, we train Laplace\-smoothed multinomial Naive Bayes heads over a fixed bag\-of\-words vocabulary \(tokens occurring in at least two training documents; maximum 7,000 terms\)\. TheFFspecialist obtainsAown=0\.951±0\.010A\_\{\\rm own\}=0\.951\\pm 0\.010,Across=0\.618±0\.034A\_\{\\rm cross\}=0\.618\\pm 0\.034, and canonicalFF\-AUC0\.949±0\.0110\.949\\pm 0\.011\. Holding the head fixed and changing only the negative\-bin mixture therefore moves the observed audit from0\.9510\.951atq=0q=0to0\.7840\.784atq=\.5q=\.5,0\.6510\.651atq=\.9q=\.9and0\.6250\.625atq=\.98q=\.98\. To instantiate Proposition[2](https://arxiv.org/html/2610.29029#Thmproposition2)as a controlled positive control, we standardize the separately trainedFFandMMscores on the training split and formsleak=0\.5​z​\(sF\)\+z⁡\(sM\)s\_\{\\rm leak\}=0\.5z\(s\_\{F\}\)\+z\(s\_\{M\}\)\. This score is worse on both own\-failure separability \(Aown=0\.886±0\.016A\_\{\\rm own\}=0\.886\\pm 0\.016\) and canonicalFF\-AUC \(0\.607±0\.0250\.607\\pm 0\.025\), but nearly perfect on the cross bin \(Across=0\.993±0\.002A\_\{\\rm cross\}=0\.993\\pm 0\.002\); the measured crossing isq∗=0\.148±0\.030q^\{\*\}=0\.148\\pm 0\.030, and the aggregate audit preferssleaks\_\{\\rm leak\}atq=\.9q=\.9in every split\.

For a support\-only refresh test we keep the correctFFlabels and vary only the number of World failures retained from the training split, leaving all non\-World examples available\. The canonicalFF\-AUC follows the same dose\-response asAownA\_\{\\rm own\}: full support gives0\.949±0\.0110\.949\\pm 0\.011, the 8\-failure arm0\.898±0\.0160\.898\\pm 0\.016, and 8\+80 replay0\.938±0\.0120\.938\\pm 0\.012\. The retained failure sets are nested within each split\. Mean held\-outAownA\_\{\\rm own\}rises smoothly with own\-failure support:0\.888±0\.0150\.888\\pm 0\.015with 2 failures,0\.895±0\.0140\.895\\pm 0\.014with 4,0\.904±0\.0140\.904\\pm 0\.014with 8,0\.917±0\.0140\.917\\pm 0\.014with 16,0\.927±0\.0110\.927\\pm 0\.011with 32,0\.937±0\.0110\.937\\pm 0\.011with 64,0\.944±0\.0100\.944\\pm 0\.010with 128, and0\.949±0\.0100\.949\\pm 0\.010with 256, versus0\.951±0\.0100\.951\\pm 0\.010under full support\. The 8\-failure arm is below full support in every split; re\-injecting 80 baseline World failures raises it to0\.941±0\.0110\.941\\pm 0\.011and improves on the collapsed refresh in every split\. These 24 deterministic splits are robustness checks, not independent replications\. Figure[15](https://arxiv.org/html/2610.29029#A30.F15)shows the audit inversion and the support\-dose curve\. The exact script, source\-file SHA and frozen summary JSON are included in the supplement; no result in this subsection was preregistered\.

Figure 15:Controlled natural\-language stress test\.Left: the same fixed component verifier looks progressively worse as its own\-failure bin vanishes, while a verifier contaminated with the other component overtakes it near the predictedq∗q^\{\*\}\. Right: with correct component labels fixed, held\-outAownA\_\{\\rm own\}degrades smoothly as own\-failure support is removed; the square marks the 8\-failure refresh plus 80 re\-injected baseline failures\. Bands/error bars show one standard deviation across 24 deterministic splits\.

## Appendix AESelection interface: audit equivalence does not imply equal influence

The audit in Section[4](https://arxiv.org/html/2610.29029#S4)uses only the ordering*within*one component score, so it is invariant to every strictly increasing reparameterizationsF↦g⁡\(sF\)s\_\{F\}\\mapsto g\(s\_\{F\}\)\. A multi\-component selector also consumes the scale ofsFs\_\{F\}relative to the other heads\. The next result characterizes exactly how much of the selector’s decision the audit leaves open\.

###### Proposition 5\(Selection ambiguity is the Pareto set\)\.

Fix a pool with component scores\(sF,sM\)\(s\_\{F\},s\_\{M\}\)and distinct score pairs \(duplicates are merged; Appendix[Y](https://arxiv.org/html/2610.29029#A25)\)\. Asggranges over strictly increasing maps—all of which leave every ROC/AUC ofsFs\_\{F\}unchanged—the set of top\-1 choices ofg⁡\(sF\)\+sMg\(s\_\{F\}\)\+s\_\{M\}is exactly the Pareto setPar\\operatorname\{Par\}of the pool in\(sF,sM\)\(s\_\{F\},s\_\{M\}\)\. Over positive rescalingsg⁡\(t\)=c​tg\(t\)=ctit is the subset ofPar\\operatorname\{Par\}on the upper convex hull\. Hence an audit statistic invariant to monotone reparameterization determines the composite decision iff\|Par\|=1\|\\operatorname\{Par\}\|=1, and determines whether that decision is on target iff all Pareto candidates share one oracle label\.

The proof is constructive \(Appendix[Y](https://arxiv.org/html/2610.29029#A25)\): a dominated candidate loses under every increasinggg, and each Pareto candidate wins under aggthat jumps at its ownsFs\_\{F\}\. This is the classical correspondence between Pareto optimality and monotone scalarization\([Miettinen, 1999](https://arxiv.org/html/2610.29029#bib.bib34)\), read here as a statement about what an audit leaves open\. Its size is predictable\.

###### Proposition 6\(Ambiguity grows with pool size\)\.

If theKKcandidates’ score pairs are i\.i\.d\. with continuous, independentsFs\_\{F\}andsMs\_\{M\}, then\|Par\|\|\\operatorname\{Par\}\|is a sum of independent Bernoulli\(1/i\)\(1/i\)variables,i=1,…,Ki=1,\\ldots,K\. Hence𝔼​\|Par\|=HK=ln⁡K\+0\.577\+o⁡\(1\)\\mathbb\{E\}\|\\operatorname\{Par\}\|=H\_\{K\}=\\ln K\+0\.577\+o\(1\)andPr\[\|Par\|=1\]=1/K\\Pr\[\|\\operatorname\{Par\}\|=1\]=1/K\. If the heads are comonotone,\|Par\|=1\|\\operatorname\{Par\}\|=1\.

Sorting candidates by decreasingsFs\_\{F\}, a candidate is Pareto iff itssMs\_\{M\}is a running maximum, which happens with probability1/i1/iindependently acrossii\([Bentley et al\., 1978](https://arxiv.org/html/2610.29029#bib.bib35)\)\. Under independent heads the audit therefore determines the best\-of\-KKdecision with probability only1/K1/K, which vanishes as test\-time compute grows; in our pools positively dependent heads slow this growth without stopping it \(Figure[16](https://arxiv.org/html/2610.29029#A31.F16)a\)\.

Figure 16:\(a\)With every component AUC fixed, the range of Answer@KKreachable by audit\-equivalent composites on the ECG base test pools grows withKK\(Proposition[6](https://arxiv.org/html/2610.29029#Thmproposition6);2020subsamples per pool\)\.\(b\)Endpoint minus hard gate against exact pair overlap for all eleven trained scalar rules \(engine 1\.5B and ECG; three seeds each; common evaluator\)\. Filled: families whose gate clears the requirement\-blind floor \(engine, Phi\); open: uninformative \(Llama near the blind pipeline, Gemma without supply\); this post\-hoc split uses only each family’s gate arm, never a rule’s gap \(Appendix[AB](https://arxiv.org/html/2610.29029#A28)\)\. All three rules with overlap≥0\.79\\geq 0\.79match the gate within0\.0070\.007\(two in uninformative families\); five of six informative low\-overlap rules miss by≥0\.08\\geq 0\.08, the exception being the contrast\-matchedw=0\.1w=0\.1\.#### How large is the ambiguity?

On the ECG base test pools atK=16K=16the Pareto set has2\.32\.3\(Llama\) and2\.22\.2\(Phi\) candidates, below the independence values2\.72\.7and2\.52\.5\(HnH\_\{n\}over thenndistinct score pairs\) that a within\-pool permutation ofsMs\_\{M\}restores\. It mixes on\- and off\-target candidates in21%21\\%and14%14\\%of pools, so with every component AUC fixed Answer@16 ranges over\[0\.33,0\.54\]\[0\.33,0\.54\]\(Llama\) and\[0\.46,0\.60\]\[0\.46,0\.60\]\(Phi\)\. These are outer bounds because oneggmust serve all pools, yet a common rescalingc​sF\+sMc\\,s\_\{F\}\+s\_\{M\}already moves Llama’s Answer@16 from0\.5000\.500\(c=0\.1c=0\.1\) to0\.3920\.392\(c=10c=10\) with supply unchanged\. A single common weight chosen on validation still lands within0\.020\.02\(Llama\) and0\.0040\.004\(Phi\) of the best common weight on test \(Appendix[AB](https://arxiv.org/html/2610.29029#A28)\): the ambiguity is resolved by validating the consumer, not by auditing the component\.

#### The engine shows the saturated extreme\.

On matched 1\.5B DPO pools, rescaling the feasibility logit byc∈\{0\.01,…,100\}c\\in\\\{0\.01,\\ldots,100\\\}leaves its AUC unchanged but moves raw\-sum best\-of\-16 from0\.5440\.544to0\.1600\.160\(Appendix[M](https://arxiv.org/html/2610.29029#A13)\); the unit sum follows the saturated logit on82%82\\%of pools \(η=0\.10\\eta=0\.10\)\. Rules that reduce its influence—a validation\-tuned weight, a soft log\-probability rule, threshold\-then\-mass—reachη=0\.88\\eta=0\.88–0\.920\.92\(0\.940\.94at 7B; Appendix[AC](https://arxiv.org/html/2610.29029#A29)\)\.

#### Takeaway\.

Component AUC fixes the order within a score; the selector’s decision is fixed only up to the Pareto set, whose size and label mix at the deployedKKshould be reported alongside any component audit\. A weight tuned for this interface is a*selection weight*; Section[AF](https://arxiv.org/html/2610.29029#A32)shows why that does not make it a training weight\.

## Appendix AFTraining interface: a good selector need not build good DPO data

Once a score constructs preference data, DPO sees only the chosen/rejected pairs, so selection quality and training\-pair quality are different objectives even for one scalar ranking rule\.

###### Proposition 7\(What reaches DPO\)\.

Fix pools, policy, reference model,β\\beta, and a deterministic rank\-based pair builder\.\(i\)The empirical DPO objective and gradient depend on the scoring rule only through the induced pair multiset; forrw=w​sF\+sMr\_\{w\}=ws\_\{F\}\+s\_\{M\}this multiset is piecewise constant inww, changing only at crossing weightswi​j=−\(sM,i−sM,j\)/\(sF,i−sF,j\)w\_\{ij\}=\-\(s\_\{M,i\}\-s\_\{M,j\}\)/\(s\_\{F,i\}\-s\_\{F,j\}\)\.\(ii\)Neither component\-order nor top\-kkequivalence implies pair equivalence when the builder uses ranks outside the topkk\.\(iii\)For uniform pair multisetsP,QP,Qof equal size,dTV​\(P,Q\)=1−overlap⁡\(P,Q\)d\_\{\\rm TV\}\(P,Q\)=1\-\\mathrm\{overlap\}\(P,Q\); if the preference log\-ratio satisfies\|dθ\|≤R\|d\_\{\\theta\}\|\\leq Rand per\-pair gradients are bounded byGG, then\|LP−LQ\|≤β​R​\(1−overlap\)\|L\_\{P\}\-L\_\{Q\}\|\\leq\\beta R\\,\(1\-\\mathrm\{overlap\}\)and‖∇LP−∇LQ‖≤2​G​\(1−overlap\)\\\|\\nabla L\_\{P\}\-\\nabla L\_\{Q\}\\\|\\leq 2G\\,\(1\-\\mathrm\{overlap\}\)\.\(iv\)If both objectives are convex andLL\-smooth, full\-batch gradient descent with stepη≤2/L\\eta\\leq 2/Lfrom a common initialization satisfies‖θTP−θTQ‖≤2​η​G​T​\(1−overlap\)\\\|\\theta^\{P\}\_\{T\}\-\\theta^\{Q\}\_\{T\}\\\|\\leq 2\\eta GT\\,\(1\-\\mathrm\{overlap\}\); withLL\-smoothness alone the factorη​T\\eta Tbecomes\(\(1\+η​L\)T−1\)/L\(\(1\+\\eta L\)^\{T\}\-1\)/L\.

Together with Proposition[5](https://arxiv.org/html/2610.29029#Thmproposition5), this gives a strict hierarchy: composite order determines the pair multiset, which determines the DPO objective; component order determines neither, and a good top\-kkselector need not fix the pairs \(Appendix[Q](https://arxiv.org/html/2610.29029#A17)gives a two\-candidate example\)\. Exact pair overlap therefore certifies closeness of the objective, of its gradient and, under \(iv\), of short training trajectories; because the trajectory bound grows with the horizon\([Hardt et al\., 2016](https://arxiv.org/html/2610.29029#bib.bib38)\), overlap certifies objectives, and endpoints only when it is near one\. Mean property contrasts are only a projection of the pair distribution with an unmeasured residual \(Proposition[4](https://arxiv.org/html/2610.29029#Thmproposition4)\)\.

#### Matched DPO exposes the mismatch\.

With the same base model, critic, train pool,4 0004\\,000pairs and recipe, the hard\-gate policy reaches0\.3110\.311target/draw and0\.7580\.758Answer@16; unit\-sum pairs reach0\.1340\.134and0\.5960\.596\(Table[39](https://arxiv.org/html/2610.29029#A32.T39)\)\. A validation pool selectsw=0\.005w=0\.005for top\-1 inference, yet training on its pairs reaches only0\.5710\.571; registered predictions that it and a soft log\-probability rule would match the gate within0\.040\.04failed\. In pair spacew=0\.005w=0\.005contrasts mass but barely feasibility, and the unit sum the opposite\.

A pre\-specifiedw=0\.1w=0\.1passes near the gate in this two\-dimensional contrast space \(Figure[8](https://arxiv.org/html/2610.29029#A17.F8), Appendix[Q](https://arxiv.org/html/2610.29029#A17)\) and reaches0\.3120\.312target/draw,0\.8210\.821supply, and0\.7700\.770Answer@16, versus0\.311/0\.807/0\.7580\.311/0\.807/0\.758for the gate\. It shares only15%15\\%of pairs with the gate, so the match is not near\-duplicate data; across the five engine rules a bottleneck contrast relative to the gate orders the endpoints \(Spearman with the endpoint0\.900\.90, versus−0\.60\-0\.60for overlap: at low overlap, overlap does not order endpoints; descriptive\), and a pre\-specified critic\-only version selectsw=0\.075w=0\.075without oracle labels \(Appendices[Q](https://arxiv.org/html/2610.29029#A17),[O](https://arxiv.org/html/2610.29029#A15)\)\.

Table 39:Matched DPO endpoints under a common fixed gate evaluator\.Δ​F\\Delta Fis chosen\-minus\-rejected feasibility andΔ​M\\Delta Mis rejected\-minus\-chosen absolute oracle log\-mass error\. “Overlap” is exact full\-pair overlap with the gate\. 7B arms are in Appendix[H](https://arxiv.org/html/2610.29029#A8)\.
#### Boundary case\.

In GSM8K the format\-score spread collapses after saturation, sum and gate pairs carry nearly identical target signal, and the trained policies agree \(0\.6470\.647vs0\.6420\.642target/draw; Appendix[C](https://arxiv.org/html/2610.29029#A3)\): the separation vanishes where the interfaces coincide\.

#### Preregistered external stress test \(ECG\)\.

Because Proposition[7](https://arxiv.org/html/2610.29029#Thmproposition7)constrains only full pair distributions, we froze an external test before training: per family, validation selectswselw\_\{\\rm sel\}and the gate\-free variant of our engine criterion \(normalize each critic contrast by its grid maximum, maximize the smaller\) selectswpairw\_\{\\rm pair\}; each rule builds4 0004\\,000pairs from one frozen train pool; three seeds; one common evaluator; margin±0\.04\\pm 0\.04\. No seed was rerun or removed\.

Table 40:Preregistered ECG training stress test \(Answer@16 under the common evaluator; mean±\\pmtraining\-seed SD\)\. Overlap is exact pair overlap with the gate’s4 0004\\,000pairs\. Last columns: the frozen primary contrast with its paired patient\-cluster bootstrap CI \(conditional on the seeds\) and a post\-hoc CI that also resamples seeds\. The single requirement\-blind pipeline scores0\.5120\.512\.
#### Frozen outcome\.

The preregistered primary outcome is negative for our criterion: pair weighting is gate\-equivalent in no family \(Table[40](https://arxiv.org/html/2610.29029#A32.T40)\)\. In Gemma it even exceeds the gate \(\+0\.119\+0\.119\), but no Gemma arm approaches the requirement\-blind pipeline \(best seed0\.2440\.244vs0\.5120\.512\), so this is not evidence for the criterion\. Gate and inference weight differ by at most0\.0070\.007with79\.679\.6–99\.5%99\.5\\%shared pairs\. With training seeds resampled together with subjects, only Phi’s pair deficit stays clear of zero \(Appendices[Z](https://arxiv.org/html/2610.29029#A26),[AA](https://arxiv.org/html/2610.29029#A27)\)\. Llama’swpair=1w\_\{\\rm pair\}=1pairs are identical to those of an added unit\-sum arm, and both fail to parse on two seeds, a shared collapse rather than a test of Proposition[7](https://arxiv.org/html/2610.29029#Thmproposition7)\.

#### What the endpoints mean \(post\-hoc\)\.

Llama arms that parse mostly emit the requirement\-blind pipeline \(0\.5120\.512\), and Gemma has near\-zero supply\. In Phi, gate and inference policies keep 79–113 distinct pipelines, whereas all pair\-weight seeds collapse onto the blind program \(≥97%\\geq 97\\%of parsed outputs\)\. The gate\-free criterion fails because in ECG the gate sits at theMM\-heavy end of the sweep, so normalizing by sweep maxima pushes towardFF; a gate\-referenced criterion would selectw=0\.003w=0\.003–0\.010\.01with8787–99%99\\%gate overlap\.

#### A trained check on CPU\.

We trained a log\-linear DPO reranker on the frozen Llama and Phi pools, where DPO is convex and Proposition[7](https://arxiv.org/html/2610.29029#Thmproposition7)\(iv\) applies exactly: gate and nine grid weights, ten seeds each, with design and four predictions hashed before any run \(Appendix[AA](https://arxiv.org/html/2610.29029#A27)\)\. All four held: the gate\-referenced weight matches the gate \(\+0\.004\+0\.004\[−0\.005,0\.015\]\[\-0\.005,0\.015\];−0\.002\-0\.002\[−0\.007,0\.002\]\[\-0\.007,0\.002\]\); Phi’s gate\-free weight is not shown equivalent and, like the LLM policies, shifts top\-1 choices onto the blind program \(67%67\\%vs25%25\\%\); across the grid\|\|gap\|\|shrinks as overlap grows \(Spearman of\|\|gap\|\|on overlap−0\.60\-0\.60,−0\.93\-0\.93\); and parameter distances respect the bound\. The reranker has little headroom above the blind program, so it corroborates direction and ordering, not LLM effect sizes\. Where gate pairs can be built, training on them is simpler than any gate\-referenced scalar\.

#### Takeaway\.

Inference scalarization is a decision rule; pre\-DPO scalarization is a*data\-construction operator*\. The evidence is one\-directional and thin \(Figure[16](https://arxiv.org/html/2610.29029#A31.F16)b, Appendix[AB](https://arxiv.org/html/2610.29029#A28)\): all three rules with≥79%\\geq 79\\%gate overlap reached the gate’s endpoint, consistent with Proposition[7](https://arxiv.org/html/2610.29029#Thmproposition7), but only Phi’s comes from a family whose gate is informative; low overlap left the endpoint undetermined\. A weak bottleneck contrast accompanied all four informative misses, but Phi’s pair weight also missed with balanced contrasts, a residual that two contrast coordinates cannot see\. In practice: when a tuned selection weight trains poorly, check its pair overlap with the rule it should emulate; if overlap is low, neither overlap nor mean contrasts certify it, and the reference rule’s own pairs are the safe default\.