Preference Is Not Intervention: The Structure and Stability Boundaries of Reader-Specific Evidence Utility

# Preference Is Not Intervention:The Structure and Stability Boundaries of Reader-Specific Evidence Utility
Source: [https://arxiv.org/html/2608.17781](https://arxiv.org/html/2608.17781)
###### Abstract

ML systems increasingly condition decisions on downstream model identity, but this is useful only if model\-specific differences form reusable structure rather than input\-local interactions\. We test this in retrieval\-augmented generation \(RAG\), where evidence utility can be measured under controlled interventions\. Holding query, evidence, task, scoring, and intervention fixed, nine readers disagree on effect sign in 33% of jointly affected cells; reader×\\timesquery interaction explains 29\.8% of utility variance versus an 8\.4% permutation null; and self\-selected evidence improves F1 by\+0\.031\+0\.031\(t=3\.39t=3\.39\)\. We then ask the sharper question:*which components of this heterogeneity are stable reader properties across queries?*Separating three measurable objects—evidence*activity*,*ordinal preference*, and*conditional signed direction*—we find ordinal reader geometry stable across four independent settings \(split\-halfρ=0\.60\\rho=0\.60–0\.830\.83\): leave\-one\-out interventions, PRISM preferences, RAMDocs, and RAGuard\. Signed geometry is task\-bounded: weak in open\-ended QA \(0\.14, 0\.35\), especially for misleading and irrelevant evidence, but strong in binary fact\-checking \(0\.75\) with no significant ordinal gap, though still below its sparsity\-matched ceiling\. Sparsity, decoding noise, and metric artifacts do not explain the main ordinal–signed gap\. Finally, stable ordinal similarity fails to predict cross\-reader intervention transfer \(oracle\-distanceρ=−0\.27\\rho=\-0\.27; regret reliability−0\.28\-0\.28\)\. Reader\-specific utility exists, but preference is not intervention: stable ranking similarity does not license transfer of help/harm decisions\.

## 1Introduction

Modern ML systems increasingly condition their behavior on the identity of the model that consumes their outputs: retrieval is personalized per generator, queries are routed per model, ensembles are weighted per member\. These bets pay off only to the extent that observed model\-specific differences contain*reusable structure*—a stable property of the model—rather than situation\-local interactions that evaporate on new inputs\. We ask when this holds, in a test bed where the question can be studied with unusual control: evidence utility in retrieval\-augmented generation \(RAG\)\.

Evidence selection in RAG is increasingly optimized for*downstream utility*: how much a document actually helps the generator, rather than how relevant it looks to a retriever\. This raises a question the literature has not cleanly answered: does utility have a systematicreader\-specificcomponent? It is established that retriever preferences and LLM\-friendly evidence diverge—the retriever–LLM preference gap\[[4](https://arxiv.org/html/2608.17781#bib.bib1)\]—and multi\-system ranking work personalizes retrieval for different RAG*agents*\[[8](https://arxiv.org/html/2608.17781#bib.bib2)\]; but agents there differ simultaneously in task, dataset, backbone, and strategy, so the effect of changing only reader identity is not identified\. What is missing is a controlled answer to:*holding query, evidence, task, and intervention fixed, does the identity of the reading model itself induce reproducible differences in passage utility—and if so, what is the structure of those differences?*

We answer both parts\.First, reader\-specific utility is real\.On 9 readers and 100 NQ/HotpotQA queries with leave\-one\-out interventions, readers disagree on whether a document helps or harms in 33% of jointly\-affected cells \(95% CI \[0\.283, 0\.377\]\); in 72% of cases where a document moves at least one reader of a pair, it moves exactly one; per\-reader nonzero\-utility rates span 13–37%; and a reader’s own measured evidence selections beat the average of other readers’ selections by\+0\.031\+0\.031F1 \(query\-clusteredt=3\.39t=3\.39\)\. Reader identity is not a nuisance variable—it is a load\-bearing axis of evidence utility\. A variance decomposition of the utility tensor sharpens the picture: reader×\\timesquery interaction accounts for 29\.8% of utility variance versus a null median of 8\.4% \(p<10−4p<10^\{\-4\}\), whereas the reader main effect is only 0\.4%\. Reader\-involving terms sum to 68% algebraically, but include an unreplicated three\-way component \(37\.1%\) that also absorbs residual variation\. The dominant reader signal is thus interaction with query–evidence context—the structure whose stability must be established, not assumed\.

Second, reader specificity is not one thing\.Having established that utilities differ across readers, we ask whether these differences constitute a*reusable reader property*—a question with direct implications for persistent reader profiles and cross\-reader transfer\. We show that “reader\-specific utility” decomposes into three measurable objects with different cross\-query stabilities:*activity*\(which documents move a reader at all\),*ordinal preference*\(how a reader ranks candidate evidence\), and*conditional signed direction*\(whether a document helps or harms, among documents that move both readers\)\. The decomposition matters because different systems consume different objects: ranking\-style supervision measures the ordinal object, while inclusion decisions depend on the signed one—a system could inherit “reader preference” structure without inheriting anything about intervention direction\.

That is exactly what we find\. Ordinal reader geometry isconsistently cross\-query stablein every setting we evaluate \(split\-half reliability 0\.60/0\.79/0\.83/0\.69 on our LOO arm, the independent PRISM preference data\[[3](https://arxiv.org/html/2608.17781#bib.bib6)\], RAMDocs, RAGuard\)\. Conditional signed geometry issubstantially weaker and task\-bounded: near the floor in open\-ended QA \(0\.14 internal; 0\.35 on RAMDocs\)—far below a sparsity\-matched stable\-world null—yet strongly stable in binary fact\-checking \(RAGuard 0\.75, no significant ordinal gap, though below its sparsity\-matched ceiling\): a boundary condition, localized to misleading and irrelevant evidence \(0\.10/0\.09\) versus partially stable supporting evidence \(0\.33\)\.

Third, the distinction has teeth downstream\.In a real cross\-reader transfer experiment \(9×\\times9 source–target pairs, 50 pre\-registered queries, 4,050 cells\), the reader specificity we establish does not assemble into transferable structure: transfer regret is predicted neither by behavioral profiles \(ρ=0\.05\\rho=0\.05\) nor by in\-sample*oracle*utility distance \(ρ=−0\.27\\rho=\-0\.27, n\.s\.\), and the regret matrix itself has no split\-half reliability \(−0\.28\-0\.28\)\. Stable ordinal similarity—the thing that*is*stable—does not license cross\-reader intervention decisions\.

#### Contributions\.

- •C1 — Controlled characterization of reader\-specific utility\.Holding query, evidence, task, scoring, and intervention fixed, we show that changing only reader identity yields substantial, structured differences in passage utility, validated behaviorally by a significant self\-reader selection advantage—to our knowledge the first controlled, multi\-reader characterization separating the*existence*of reader heterogeneity from the*stability*of its components \(§[5\.1](https://arxiv.org/html/2608.17781#S5.SS1)\)\.
- •C2 — Stability decomposition and its boundary\.Evidence activity and ordinal preference geometry are consistently cross\-query stable across four independent settings; conditional help/harm geometry is substantially weaker, task\-bounded, and localized to misleading/noise evidence in open\-ended QA; frozen calibrations show that sparsity, decoding, and metric artifacts do not explain the main ordinal–signed gap \(§[5\.2](https://arxiv.org/html/2608.17781#S5.SS2)–§[5\.4](https://arxiv.org/html/2608.17781#S5.SS4)\)\. A matched forced\-choice perturbation gives suggestive evidence that constraining the answer space stabilizes responses to misleading evidence \(§[5\.5](https://arxiv.org/html/2608.17781#S5.SS5)\)\.
- •C3 — Practical consequence\.Stable ordinal reader similarity does not provide a stable basis for cross\-reader intervention transfer \(§[5\.6](https://arxiv.org/html/2608.17781#S5.SS6)\)\.

## 2Related Work

#### Consumer\-dependent retrieval and downstream utility\.

[4](https://arxiv.org/html/2608.17781#bib.bib1)establish that retriever relevance preferences diverge from what downstream LLMs can exploit; multi\-agent ranking work personalizes retrieval for 18 RAG agents from downstream feedback\[[8](https://arxiv.org/html/2608.17781#bib.bib2);[7](https://arxiv.org/html/2608.17781#bib.bib3)\]; R3AG\[[15](https://arxiv.org/html/2608.17781#bib.bib4)\]routes retrievers per generator\. In all of these, consumers differ simultaneously in task, dataset, backbone, and strategy, so reader identity is confounded with task; we isolate it with everything else fixed\.

#### LLM\-specific utility notions\.

Concurrent preprint work proposes LLM\-specific passage utility and reports limited transfer of utilitarian passages across generators\[[14](https://arxiv.org/html/2608.17781#bib.bib5)\]\. We independently establish reader\-conditioned utility differences under controlled interventions, but ask a different question: which components of the heterogeneity constitute stable reader structure across queries\. Rank4Gen\[[3](https://arxiv.org/html/2608.17781#bib.bib6)\]learns generator\-conditioned ranking from PRISM, which we use as an independent external replication resource for the ordinal component\. Generator\-agnostic utility rerankers that transfer across readers\[[11](https://arxiv.org/html/2608.17781#bib.bib7);[1](https://arxiv.org/html/2608.17781#bib.bib8)\]are consistent with our decomposition: predictable utility lives largely on the query/evidence side and in activity/ordinal structure\.

#### Conflict robustness and methodology\.

RAMDocs\[[9](https://arxiv.org/html/2608.17781#bib.bib9)\]and RAGuard\[[13](https://arxiv.org/html/2608.17781#bib.bib10)\]supply typed evidence \(supporting / misleading / noise\); MAGIC\[[6](https://arxiv.org/html/2608.17781#bib.bib11)\]benchmarks inter\-context conflict resolution\. This literature asks whether models*answer correctly*under conflict; we ask whether the*direction*of a document’s effect is a stable reader attribute—and find the instability concentrates on exactly these adversarial evidence types\. Our protocol builds on classical split\-half reliability with stratified splits and sparsity\-matched permutation calibration\.

## 3Three Stabilities of Reader\-Specific Utility

### 3\.1Readers and utility

Areaderis a model endpoint under a fixed deployment configuration—model, decoding policy, and serving stack fixed for the duration of the study—not an architecture\-intrinsic personality\. This makes reader identity operational and is what transfer systems actually condition on in deployment\.

Autility operatormaps \(readermm, queryqq, documentdd\) to a scalarU⁡\[m,q,d\]U\[m,q,d\]relative to a baseline\. We use two operators:

LOO:U⁡\[m,q,d\]=scorem​\(q,D\)−scorem​\(q,D∖\{d\}\),\\displaystyle U\[m,q,d\]=\\mathrm\{score\}\_\{m\}\(q,D\)\-\\mathrm\{score\}\_\{m\}\(q,D\\setminus\\\{d\\\}\),\(1\)Single\-doc:U⁡\[m,q,d\]=scorem​\(q,\{d\}\)−scorem​\(q,∅\),\\displaystyle U\[m,q,d\]=\\mathrm\{score\}\_\{m\}\(q,\\\{d\\\}\)\-\\mathrm\{score\}\_\{m\}\(q,\\varnothing\),\(2\)whereDDis a top\-kkretrieved context and∅\\varnothingis the closed\-book condition\. Scores are deterministic task metrics \(token\-F1 or exact/binary match, per dataset protocol\)\.

### 3\.2Three measurable objects

From the utility tensor we derive three reader\-pair geometries\. For readersi,ji,jand queryqq:

- •Activity: full\-support sign agreement, counting the zero pattern as signal—whether a document moves the reader at all\.
- •Ordinal preference: Spearman correlation between the two readers’ utility vectors onqq—relative preference, the object that preference\-pair and listwise supervision captures\.
- •Conditional signed direction: sign agreement restricted to documents nonzero for*both*readers \(dual\-nonzero cells\)—intervention direction: help versus harm\.

Each object induces a per\-query distance betweeniiandjj; aggregating over queries \(median\) yields a reader\-pair distance matrixDD—the object’s*reader geometry*\. Figure[1](https://arxiv.org/html/2608.17781#S3.F1)illustrates why the objects must be separated: two readers can rank six documents identically \(Spearman=1=1\) while disagreeing on the sign of a third of them\.

Figure 1:Same ranking, different zero\-crossings \(illustrative\)\.Ordinal agreement between two readers can coexist with opposite help/harm signs\. Relative\-preference supervision measures the former; inclusion decisions depend on the latter\.
### 3\.3Stability as split\-half reliability of the geometry

An object’s cross\-querystabilityis the split\-half reliability of its geometry: split the queries into two stratified halves \(by dataset source and, where applicable, gold verdict\), computeDDon each half, and correlate the two distance vectors across reader pairs \(Spearmanρ\\rho; 1,000 random splits; we report the median and 2\.5/97\.5 percentiles\)\. A stable reader property yieldsDA≈DBD\_\{A\}\\approx D\_\{B\}; a query\-local interaction yields uncorrelated halves\. Pairwise geometry is the weakest object that similarity\-based personalization relies on, and it makes no parametric commitment about reader profiles; whether it suffices for profile\-level prediction is tested directly in §[5\.6](https://arxiv.org/html/2608.17781#S5.SS6)\.

### 3\.4Permutation calibration against sparsity

Signed utility is sparse: most documents do not change most readers’ scores, so dual\-nonzero cells are rare, and*some*loss of split\-half reliability is expected from sparsity alone\. To separate measurement sparsity from genuine instability, every signed estimate is calibrated against an arm\-specificstable\-world permutation\. All variants preserve the observed sparse support and relevant conflict\-count marginals while imposing stable reader\-pair conflict propensities across queries\. The internal arm permutes observed conflict indicators within each reader pair; the external arms permute them within reader\-pair×\\timesevidence\-position strata \(2,000– 5,000 simulations\)\. Appendix[E](https://arxiv.org/html/2608.17781#A5)gives both constructions\. Observedρ\\rhofar below the null median rejects “sparsity explains the weakness\.” A parallel null calibrates ordinal stability against type\-level structure alone; repeated\-decoding runs bound decoding stochasticity \(§[6](https://arxiv.org/html/2608.17781#S6)\)\.

### 3\.5What stability would license

If conditional signed geometry were stable, a reader’s help/harm pattern estimated on one query set would transfer to new queries, reader profiles could be built once and reused, and reader\-conditioned selection could target intervention direction\. If only ordinal geometry is stable, ranking\-style reader conditioning may still work—but inclusion decisions and cross\-reader transfer of intervention choices are not licensed by the same evidence\. Section[5\.6](https://arxiv.org/html/2608.17781#S5.SS6)tests the transfer consequence directly\.

## 4Experimental Setup

### 4\.1Readers

The internal LOO arm uses9 core readers: five API endpoints \(Qwen3\.6\-Flash, DeepSeek\-V4\-Flash, GLM\-5\.2, GPT\-5\.6\-Luna, K3\) and four local 8–9B GGUF endpoints \(Qwen3\.5\-9B\-Instruct, Ministral\-8B\-Instruct, Llama\-3\.3\-8B\-Instruct, Llama\-3\.1\-8B\-Instruct\) served on a single RTX\-4060 host \(full roster and decoding configurations: Appendix[A](https://arxiv.org/html/2608.17781#A1)\)\. The external single\-document arm uses13 readers: the 9 core plus Qwen3\.7\-Plus, Qwen3\.7\-Max, Qwen3\.8\-Max, and DeepSeek\-V4\-Pro\. Decoding is deterministic wherever the endpoint permits \(one endpoint requires temperature 1\.0 with forced reasoning; it is treated as a distinct deployment configuration per §[3](https://arxiv.org/html/2608.17781#S3)\)\. Local endpoints are quantized builds; the design compares*relative geometry across readers*, not absolute capability tiers, and scale\-strata cuts are reported as descriptive only\.

### 4\.2Internal arm: LOO utility on NQ/HotpotQA

100 queries \(50 NQ\[[5](https://arxiv.org/html/2608.17781#bib.bib12)\], 50 HotpotQA\[[12](https://arxiv.org/html/2608.17781#bib.bib13)\]\), each with 8 unique BGE\-M3\-ranked\[[2](https://arxiv.org/html/2608.17781#bib.bib14)\]candidate documents with guaranteed supporting evidence\. Every reader answers 10 conditions per query \(closed\-book, full 8\-document context, and eight leave\-one\-out contexts\), scored by token\-level F1 against gold answers: a9×100×89\\times 100\\times 8utility tensor \(7,200 cells\)\. Prompt templates and scoring contracts for all arms are given in Appendix[B](https://arxiv.org/html/2608.17781#A2)\.

### 4\.3External arms: single\-document interventions

RAMDocs\[[9](https://arxiv.org/html/2608.17781#bib.bib9)\]\(open\-ended disambiguation QA with misinformation\): 149 queries meeting a frozen eligibility rule—at least two supporting \(correct\), one misleading \(misinformation\), and one noise document—with documents selected by frozen data order\. Each \(reader, query\) contributes 5 conditions: closed\-book plus four single\-document contexts \(support1, support2, mislead1, noise1\)\. Scoring follows the dataset’s strict protocol, adapted to single\-document conditions as normalized any\-gold match\.

RAGuard\[[13](https://arxiv.org/html/2608.17781#bib.bib10)\]\(real\-world fact verification against misleading Reddit retrievals\): 212 claims under the same eligibility rule; scoring is binary verdict match with deterministic parsing\. Both arms stratify query splits by dataset and \(for RAGuard\) gold verdict\. Per\-position signed analyses pool the two support positions and report the misleading and noise positions separately\.

### 4\.4External ordinal arm: PRISM

PRISM/Rank4Gen\-DPO public data\[[3](https://arxiv.org/html/2608.17781#bib.bib6)\]: 58,404 preference rows→\\to29,197 unique \(query, generator\) keys \(bilingual prompt variants deduplicated\), covering 7,791 unique queries across five sources \(HotpotQA, 2WikiMultiHopQA, MuSiQue, MS MARCO, CRUD\-RAG\) and seven downstream generators\. Each key’s*chosen*ordered document set is the generator’s preferred context\. Candidate indices are translated to per\-query canonical document ids via normalized\-text digests \(99\.98% pool\-Jaccard across generators\), so cross\-generator comparisons are of the same documents\. Ordinal geometry: per\-query RBO\[[10](https://arxiv.org/html/2608.17781#bib.bib15)\]\(p=0\.9p=0\.9\) between chosen ordered sequences; composition\-only auxiliary via Jaccard; 21 generator pairs; the same 1,000\-split stratified protocol\. PRISM has no signed operator—it calibrates the ordinal half only\.

#### Freezing and reproducibility\.

Query eligibility rules, per\-arm decision rules, split counts and seeds, parsing tiers, and the transfer plan were frozen before results analysis; artifact\-level checksums and frozen plans are retained \(§[J](https://arxiv.org/html/2608.17781#A10)\)\. All analyses run on frozen artifacts with fixed seeds\.

## 5Results

### 5\.1RQ1 — Reader\-specific utility is real \(controlled heterogeneity\)

Figure 2:Changing only the reader changes which evidence matters\.Across 9 readers and 100 fixed NQ/HotpotQA queries, \(a\) activity spans 13\.4–36\.6%, \(b\) mean retrieval gain is positive for every reader, and \(c\) 72\.3% of affected pair\-cells move only one reader while 33\.3% reverse the help/harm sign when both move\.Before stability, existence\. With everything else fixed, changing only reader identity yields substantial differences \(Figure[2](https://arxiv.org/html/2608.17781#S5.F2); per\-reader statistics in Appendix[C](https://arxiv.org/html/2608.17781#A3)\)\. All readers benefit from retrieval on average \(gain\+0\.169\+0\.169to\+0\.386\+0\.386F1\), but*which*documents drive the gain differs sharply:

- •Sign disagreement\.Pooling all 36 reader pairs, readers assign*opposite signs*to a document’s effect in33\.3%of dual\-nonzero cells \(1,067/3,206; query\-cluster bootstrap 95% CI \[0\.283, 0\.377\]\)\.
- •Activity asymmetry\.In72\.3%of \(pair, cell\) cases where a document moves at least one reader of a pair, it moves exactly one \(8,348/11,554\)\. Per\-reader nonzero rates span 13\.4–36\.6% \(2\.7×\\times\)\. The external arms show the same spread \(RAGuard: 16\.8–51\.5%\)\.
- •Behavioral validation\.A reader’s own measured preference set outperforms the average of other readers’ sets by\+0\.031\+0\.031F1\(query\-clusteredt=3\.39t=3\.39, df=49=49; in sample, §[5\.6](https://arxiv.org/html/2608.17781#S5.SS6)\) — reader\-specific utility is not only measurable but actionable in principle\.

A three\-way variance decomposition of the 7,200\-cell tensor \(reader×\\timesquery×\\timesrank position; Appendix[C](https://arxiv.org/html/2608.17781#A3)\) makes the same point at the distribution level: the clean reader×\\timesquery interaction accounts for 29\.8% of variance versus a permutation\-null median of 8\.4% \(p<10−4p<10^\{\-4\}\), while the reader main effect is only 0\.4%\. All reader\-involving terms sum to 68\.0% of the algebraic decomposition\. Because there is one observation per reader×\\timesquery×\\timesposition cell, that sum includes a 37\.1% three\-way component that also absorbs residual variation\. The signal is nevertheless large and interaction\-led, not a uniform reader\-level shift\.

Heterogeneity of this size is what reader\-conditioned evidence selection hopes to exploit\. The rest of the paper asks what part of it is a stable reader property\.

### 5\.2RQ2 — Ordinal preference geometry is consistently stable

Figure 3:Ordinal stability is broad; signed stability has a task boundary\.Cross\-query split\-half reliability of reader\-pair geometry across four independent settings \(points: medians over 1,000 stratified splits; whiskers: 2\.5/97\.5 percentiles\)\. Blue: ordinal geometry\. Red: conditional signed geometry\. Gray ticks: sparsity\-matched stable\-world nulls \(§[3\.4](https://arxiv.org/html/2608.17781#S3.SS4)\)\. PRISM has no signed operator\.Ordinal reader geometry reproduces across disjoint query halves in every evaluated setting \(Figure[3](https://arxiv.org/html/2608.17781#S5.F3), blue\): 0\.599 \[0\.370, 0\.764\] on the internal LOO arm, 0\.786 \[0\.660, 0\.879\] on PRISM, 0\.833 \[0\.676, 0\.934\] on RAMDocs, and 0\.685 \[0\.571, 0\.781\] on RAGuard—spanning two utility operators, two answer formats, and an independent preference pipeline with different generators and query sources\. On PRISM the result survives artifact controls: an identity\-shuffle null \(chosen\-list lengths preserved, document identities resampled\) yields only 0\.314 against the observed 0\.786; an order\-shuffle null yields 0\.703, so composition carries most of the stability and ordering adds∼\\sim0\.08; an exactly size\-matched comparison is*more*stable, not less \(0\.862\); and composition\-only Jaccard geometry is likewise stable \(0\.742\)\. Full split distributions and null calibrations are in Appendices[D](https://arxiv.org/html/2608.17781#A4)and[E](https://arxiv.org/html/2608.17781#A5)\.

### 5\.3RQ3 — Conditional signed geometry is weaker, with a task boundary

Table 1:Conditional signed vs\. ordinal stability and stable\-world nulls\.In open\-ended QA, signed geometry lies far below ordinal geometry and its sparsity\-matched ceiling\. In binary fact\-checking it is statistically indistinguishable from ordinal geometry, although still below that ceiling\.Three observations \(Figure[3](https://arxiv.org/html/2608.17781#S5.F3), red; Table[1](https://arxiv.org/html/2608.17781#S5.T1)\)\.First, in open\-ended QA the conditional signed geometry is far below the ordinal geometry—paired per\-splitΔ=0\.487\\Delta=0\.487\[0\.226, 0\.721\] on RAMDocs—and far below what measurement sparsity alone predicts: the stable\-world null expectsρ≈0\.36\\rho\\approx 0\.36–0\.380\.38under identical support, while observed values are 0\.14 and 0\.35 \(p=×10−4p=2\\\!\\times\\\!10^\{\-4\}and×10−45\\\!\\times\\\!10^\{\-4\}; two further calibrations concur, Appendix[E](https://arxiv.org/html/2608.17781#A5)\)\.Second, the weakness is not universal: in binary fact\-checking the signed geometry is as stable as the ordinal one \(pairedΔ=−0\.064\\Delta=\-0\.064\[−0\.201\-0\.201, 0\.114\]\)\. It is strongly stable, not at the stable\-world ceiling \(0\.748 vs\. 0\.814,p=×10−4p=5\\\!\\times\\\!10^\{\-4\}\)\. Our own data therefore reject the universal instability claim and instead establish atask boundary\.Third, both external arms use a different intervention operator than the internal arm, so the open\-QA weakness is not an artifact of leave\-one\-out redundancy\. The signed reading is “substantially weaker than ordinal and below its sparsity\-matched ceiling,” not “zero\.”

### 5\.4RQ4 — In open\-ended QA, instability localizes to misleading and noise evidence

Figure 4:The task boundary localizes to evidence type\.Points and whiskers show per\-position signed split\-half reliability and its 2\.5/97\.5 percentiles\. RAMDocs \(circles\) is weak for misleading and noise evidence; RAGuard \(squares\) is stable across all three positions\.RAMDocs per\-position signed stability \(Figure[4](https://arxiv.org/html/2608.17781#S5.F4)\): supporting evidence 0\.330 \[0\.080, 0\.536\] over 14,549 dual\-nonzero cells; misleading 0\.104 \[−0\.137\-0\.137, 0\.343\] over 2,561; noise 0\.093 \[−0\.107\-0\.107, 0\.279\] over 1,888\. The weakness is not uniform across evidence processing: directions of*supporting*evidence are partially stable, while directions of*misleading*and*irrelevant*evidence are essentially query\-local\. RAGuard shows the opposite pattern \(0\.658 / 0\.713 / 0\.377\)—in binary verification, even misleading\-evidence direction is stable across queries\. What varies by task regime is specifically the cross\-query stability ofhow a reader responds to adversarial evidence\.

### 5\.5A matched mechanism probe — forced choice \(suggestive\)

Figure 5:Constraining the answer space \(matched perturbation, RAMDocs arm\)\.\(a\) Open\-ended vs\. forced\-choice signed stability, overall and by evidence type: the elevation concentrates on misleading evidence \(0\.10→0\.590\.10\\to 0\.59\)\. \(b\) Label\-stratum asymmetry: stabilizing geometry measurable in the gold = A stratum; gold = B degenerate\.One candidate mechanism is the answer space: binary verdicts leave less room for reader\-specific conflict resolution than open\-ended answers\. We test this with a matched RAMDocs perturbation that changes only the output contract to a two\-option forced choice \(gold vs\. misinformation\-supported answer; fixed A/B assignment\), using the pre\-registered decision rule in Appendix[G](https://arxiv.org/html/2608.17781#A7)\.

Overall signed stability rises from 0\.345 to 0\.479 \(pairedΔ=\+0\.130\\Delta=\+0\.130\[−0\.114\-0\.114, 0\.403\]\), partial evidence under the frozen rule\. The elevation is misleading\-selective \(0\.104 to 0\.594\), while supporting evidence is unchanged \(0\.330 to 0\.257\)\. The forced\-side mislead geometry is measurable only in the gold = A stratum \(0\.599; gold = B degenerate\), and remains below its own stable null \(0\.516,p=×10−4p=5\\\!\\times\\\!10^\{\-4\}\)\. We therefore treat the answer\-space account as asuggestive mechanism, not an identified cause; the task boundary itself is the established result\.

### 5\.6Practical consequence — stable similarity does not license transfer

Figure 6:Stable similarity does not predict intervention transfer\.\(a\) Across 36 reader pairs, in\-sample oracle utility distance is unrelated to symmetrized transfer regret \(ρ=−0\.27\\rho=\-0\.27,p=0\.264p=0\.264\)\. \(b\) The regret matrix has no reliable split\-half structure \(ρ=−0\.28\\rho=\-0\.28, 95% interval \[−0\.53\-0\.53,0\.000\.00\]\); self\-selected evidence still gives the in\-sample\+0\.031\+0\.031F1 advantage reported in the text\.If stable ordinal geometry licensed intervention\-level personalization, reader similarity should predict transfer\. We test this on 9 target readers and 50 pre\-registered queries using tie\-aware source preference sets, yielding 4,050 source×\\timestarget×\\timesquery cells \(construction details: Appendix[H](https://arxiv.org/html/2608.17781#A8)\)\.

Reader\-specific utility is real \(in sample\):self\-selected evidence beats other readers’ sets by\+0\.031\+0\.031F1 \(t=3\.39t=3\.39\)\.But similarity does not predict transfer:oracle utility distance vs\. regret givesρ=−0\.271\\rho=\-0\.271\(p=0\.264p=0\.264\), and the regret matrix has no split\-half reliability \(−0\.281\-0\.281\[−0\.530\-0\.530, 0\.001\]; mean regret 0\.035 F1, mean standard error 0\.037\)\.

Stable ordinal similarity is therefore not a sufficient basis for cross\-reader intervention decisions, even when similarity is measured in sample\.

## 6Alternative Explanations

Table 2:Alternative explanations and their disposition\.Table[2](https://arxiv.org/html/2608.17781#S6.T2)summarizes the controls\. The test–retest analysis \(Appendix[F](https://arxiv.org/html/2608.17781#A6)\) shows that decoding stochasticity is real but does not explain the main ordinal–signed gap: the attenuation\-corrected ceiling remains far below ordinal stability on both the internal and RAMDocs arms\.

## 7Discussion

#### What kind of reader specificity exists?

Our results separate three claims that “reader\-specific utility” collapses into one\. Readers genuinely differ on matched query–evidence conditions, and consequentially \(§[5\.1](https://arxiv.org/html/2608.17781#S5.SS1)\)\. The*ordinal*reading is additionally stable: it reproduces across disjoint query sets in all four settings, including an independent preference pipeline\. The*signed*reading is not uniformly stable: weak in open\-ended QA, concentrated on adversarial evidence, strong in binary verification\. “Utility is reader\-specific” is true per query, yet not a stable signed trait across queries in open\-ended QA\.

#### Reconciling generator\-conditioned and generator\-agnostic methods\.

Generator\-conditioned ranking can work because ordinal/compositional preference geometry is genuinely cross\-query stable; generator\-agnostic utility rerankers can generalize because much predictable utility lives on the query/evidence side and in activity structure\. Both are compatible with signed direction being query\-local: neither ranking nor transferable utility estimation requires stable help/harm geometry\.

#### For personalization systems\.

Condition on the reader for*ranking and composition*, where stability holds\. For*intervention\-level*personalization, stable ordinal similarity implies neither transferable intervention decisions nor a reliably structured transfer outcome: treat help/harm direction as query\-local unless the task regime is known to stabilize it\.

#### For evaluation practice\.

Preference\-pair and listwise supervision measure ordinal structure; deployment decisions depend on intervention direction\. A method can look stable under ranking\-style evaluation while the property that matters for inclusion decisions is query\-local\. Cross\-query reliability of*signed*utility—with sparsity\-matched calibration—should accompany any claim of stable reader preference\. Our data suggest the boundary hypothesis that intervention direction stabilizes when the task constrains how evidence can be realized in the answer; identifying the causal axis is left to future work\.

## 8Limitations

The task boundary is established, but its causal axis is not identified: the forced\-choice perturbation is partial evidence with label\-stratum degeneracy\. We do not claim priority for model\-dependent passage utility \(a concurrent preprint proposes the notion\[[14](https://arxiv.org/html/2608.17781#bib.bib5)\]\); ours is the controlled characterization and stability decomposition\. The two arms use different reader panels and PRISM has no signed operator; cross\-setting comparisons are of stability patterns, not identical measurements\. Transfer uses 50 queries per cell; utility is task\-metric\-based\.

## 9Conclusion

Reader\-specific evidence utility is real and consequential: with all else fixed, changing the reader changes which documents help, harm, or do nothing\. Relative evidence preferences are consistently cross\-query stable across readers, datasets, and preference constructions; the stability of help\-versus\-harm directions is task\-bounded—weaker in open\-ended QA, concentrated on misleading and irrelevant evidence, and absent as a gap in binary fact\-checking\. Preference is not intervention: stable ordinal geometry licenses neither cross\-reader transfer nor claims of stable help/harm direction\.

## References

- \[1\]M\. Chandra, D\. Ganguly, and I\. Ounis\(2026\)LURE\-RAG: lightweight utility\-driven reranking for efficient RAG\.arXiv preprint arXiv:2601\.19535\.Cited by:[§2](https://arxiv.org/html/2608.17781#S2.SS0.SSS0.Px2.p1.1)\.
- \[2\]J\. Chen, S\. Xiao, P\. Zhang, K\. Luo, D\. Lian, and Z\. Liu\(2024\)M3\-embedding: multi\-linguality, multi\-functionality, multi\-granularity text embeddings through self\-knowledge distillation\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 2318–2335\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.137)Cited by:[§4\.2](https://arxiv.org/html/2608.17781#S4.SS2.p1.1)\.
- \[3\]Y\. Fan, Y\. Chu, Z\. Xia, X\. Chen, J\. Liu, H\. Liang, J\. Ma, B\. He, Y\. Sun, J\. Zhai, D\. Ye, and T\. Ruan\(2026\)Rank4Gen: RAG\-preference\-aligned document set selection and ranking\.arXiv preprint arXiv:2601\.11273\.Cited by:[§1](https://arxiv.org/html/2608.17781#S1.p5.1),[§2](https://arxiv.org/html/2608.17781#S2.SS0.SSS0.Px2.p1.1),[§4\.4](https://arxiv.org/html/2608.17781#S4.SS4.p1.1)\.
- \[4\]Z\. Ke, W\. Kong, C\. Li, M\. Zhang, Q\. Mei, and M\. Bendersky\(2024\)Bridging the preference gap between retrievers and LLMs\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 10438–10451\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.562)Cited by:[§1](https://arxiv.org/html/2608.17781#S1.p2.1),[§2](https://arxiv.org/html/2608.17781#S2.SS0.SSS0.Px1.p1.1)\.
- \[5\]T\. Kwiatkowski, J\. Palomaki, O\. Redfield, M\. Collins, A\. Parikh, C\. Alberti, D\. Epstein, I\. Polosukhin, J\. Devlin, K\. Lee, K\. Toutanova, L\. Jones, M\. Kelcey, M\. Chang, A\. M\. Dai, J\. Uszkoreit, Q\. Le, and S\. Petrov\(2019\)Natural questions: a benchmark for question answering research\.Transactions of the Association for Computational Linguistics7,pp\. 453–466\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00276)Cited by:[§4\.2](https://arxiv.org/html/2608.17781#S4.SS2.p1.1)\.
- \[6\]J\. Lee, K\. Lee, and T\. Kim\(2025\)MAGIC: a multi\-hop and graph\-based benchmark for inter\-context conflicts in retrieval\-augmented generation\.arXiv preprint arXiv:2507\.21544\.Cited by:[§2](https://arxiv.org/html/2608.17781#S2.SS0.SSS0.Px3.p1.1)\.
- \[7\]A\. Salemi and H\. Zamani\(2024\)Towards a search engine for machines: unified ranking for multiple retrieval\-augmented large language models\.InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval,pp\. 741–751\.External Links:[Document](https://dx.doi.org/10.1145/3626772.3657733)Cited by:[§2](https://arxiv.org/html/2608.17781#S2.SS0.SSS0.Px1.p1.1)\.
- \[8\]A\. Salemi and H\. Zamani\(2025\)Learning to rank for multiple retrieval\-augmented models through iterative utility maximization\.InProceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval,pp\. 183–193\.External Links:[Document](https://dx.doi.org/10.1145/3731120.3744584)Cited by:[§1](https://arxiv.org/html/2608.17781#S1.p2.1),[§2](https://arxiv.org/html/2608.17781#S2.SS0.SSS0.Px1.p1.1)\.
- \[9\]H\. Wang, A\. Prasad, E\. Stengel\-Eskin, and M\. Bansal\(2025\)Retrieval\-augmented generation with conflicting evidence\.arXiv preprint arXiv:2504\.13079\.Cited by:[§2](https://arxiv.org/html/2608.17781#S2.SS0.SSS0.Px3.p1.1),[§4\.3](https://arxiv.org/html/2608.17781#S4.SS3.p1.1)\.
- \[10\]W\. Webber, A\. Moffat, and J\. Zobel\(2010\)A similarity measure for indefinite rankings\.ACM Transactions on Information Systems28\(4\),pp\. 1–38\.External Links:[Document](https://dx.doi.org/10.1145/1852102.1852106)Cited by:[§4\.4](https://arxiv.org/html/2608.17781#S4.SS4.p1.1)\.
- \[11\]Y\. Wu, X\. Shen, F\. Wang, C\. Zhou, Z\. Wu, X\. Dai, and R\. Xia\(2026\)Optimizing RAG rerankers with LLM feedback via reinforcement learning\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),External Links:[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.1406)Cited by:[§2](https://arxiv.org/html/2608.17781#S2.SS0.SSS0.Px2.p1.1)\.
- \[12\]Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. W\. Cohen, R\. Salakhutdinov, and C\. D\. Manning\(2018\)HotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,pp\. 2369–2380\.External Links:[Document](https://dx.doi.org/10.18653/v1/D18-1259)Cited by:[§4\.2](https://arxiv.org/html/2608.17781#S4.SS2.p1.1)\.
- \[13\]L\. Zeng, R\. Gupta, D\. Motwani, Y\. Zhang, and D\. Yang\(2025\)Worse than zero\-shot? a fact\-checking dataset for evaluating the robustness of RAG against misleading retrievals\.InAdvances in Neural Information Processing Systems 38: Datasets and Benchmarks Track,pp\. 179845–179871\.External Links:[Document](https://dx.doi.org/10.52202/085713-5409)Cited by:[§2](https://arxiv.org/html/2608.17781#S2.SS0.SSS0.Px3.p1.1),[§4\.3](https://arxiv.org/html/2608.17781#S4.SS3.p2.1)\.
- \[14\]H\. Zhang, K\. Bi, J\. Guo, J\. Zhang, S\. Wang, D\. Yin, and X\. Cheng\(2025\)LLM\-specific utility: a new perspective for retrieval\-augmented generation\.arXiv preprint arXiv:2510\.11358\.Cited by:[§2](https://arxiv.org/html/2608.17781#S2.SS0.SSS0.Px2.p1.1),[§8](https://arxiv.org/html/2608.17781#S8.p1.1)\.
- \[15\]T\. Zhao, Y\. Zhu, Y\. Tian, and Z\. Dou\(2026\)R3R^\{3\}AG: retriever routing for retrieval\-augmented generation\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),External Links:[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.939)Cited by:[§2](https://arxiv.org/html/2608.17781#S2.SS0.SSS0.Px1.p1.1)\.

## Ethics and Reproducibility

All datasets are public \(NQ, HotpotQA, RAMDocs, RAGuard, PRISM\) and used within their licenses; no human subjects or personal data are involved\. RAGuard documents originate from Reddit; we use them only as retrieved evidence under the dataset’s terms\. All experimental decisions reported here were frozen before results analysis; artifacts, frozen plans, and analysis scripts will be released\. Misleading\-evidence findings describe model behavior under misinformation and carry no endorsement of the misleading content\.

Provenance\.An earlier candidate reader failed the response\-validity criterion and was excluded before the final analysis panel; primary conclusions were recomputed on the finalized roster\.

## AI Usage Disclosure

AI assistants were used for literature cross\-checking, feedback on framing and experimental design, interpretation of results, and manuscript editing\. All experimental decision rules were frozen before results analysis, all analyses were executed programmatically on frozen artifacts, and every reported number was recomputed from those artifacts\. The authors take full responsibility for all content\.

## Appendix AReader roster and deployment configurations

Table[3](https://arxiv.org/html/2608.17781#A1.T3)lists the 13 readers\. Following §[3](https://arxiv.org/html/2608.17781#S3), a*reader*is a model endpoint under a fixed deployment configuration\. All endpoints except K3 run deterministic decoding \(temperature 0, reasoning disabled, 128\-token cap\)\. K3’s endpoint forces temperature 1\.0 and always\-on reasoning; we run it with the minimal reasoning effort and a 2,048\-token budget and analyze it as its own deployment configuration\. Local readers are GGUF builds served on a single RTX\-4060 host\. Internal identifiers predate the final display names:reader\_qwen\_flash,reader\_deepseek\_flash, andreader\_glmdenote Qwen3\.6\-Flash, DeepSeek\-V4\-Flash, and GLM\-5\.2 respectively\.

Table 3:Reader roster\.Arms: LOO==internal leave\-one\-out arm \(9 readers\); SD==external single\-document arms \(13 readers\)\.∗Endpoint constraint: K3 permits neither lower temperatures nor disabling reasoning; it is analyzed as a distinct deployment configuration\.The pre\-registered exclusion rule \(fallback/parse failure\>25%\>25\\%of a reader’s conditions, or nonzero utility rate<5%<5\\%\) was applied mechanically per arm\. No reader triggered it in any final arm; the highest fallback rate is 21\.6% \(Ministral\-8B on RAMDocs\) and the next highest 7\.8%\. Per\-reader QC statistics are recorded in the analysis artifacts \(Appendix[J](https://arxiv.org/html/2608.17781#A10)\)\. Scale strata used for descriptive cuts on the single\-document arm:*api\-mid*==\{Qwen3\.6\-Flash, Qwen3\.7\-Plus, DeepSeek\-V4\-Flash\};*frontier*==\{Qwen3\.7\-Max, Qwen3\.8\-Max, DeepSeek\-V4\-Pro, GLM\-5\.2, GPT\-5\.6\-Luna, K3\};*local\-small*==the four local readers\.

## Appendix BPrompts and scoring contracts

All QA\-style conditions—every LOO condition, the RAMDocs and RAGuard single\-document conditions, and all transfer evaluations—share one prompt contract:

> System:You are answering a factual question\. Follow the output format exactly\. User:Question:⟨\\langlequestion⟩\\rangle Evidence: \[1\]⟨\\langledocument 1⟩\\rangle; \[2\]⟨\\langledocument 2⟩\\rangle; …; \[kk\]⟨\\langledocument k⟩\\rangle Return only: Answer:⟨\\langleshort answer⟩\\rangle

Closed\-book conditions replace the evidence block with “\(No evidence provided\.\)”\. RAGuard questions are wrapped as “Is the following claim true or false? Claim: ‘⟨\\langleclaim⟩\\rangle’ ” with gold answersTrue/False; document text is the title plus the first 4,000 characters of the full text \(97\.2% of selected documents covered in full\)\.

Forced\-choice menu\(§[5\.5](https://arxiv.org/html/2608.17781#S5.SS5); RAMDocs arm only\):

> ⟨\\langlequestion⟩\\rangle Which of the two options is the correct answer to the question above? A\)⟨\\langleoption A⟩\\rangle B\)⟨\\langleoption B⟩\\rangle Respond with the option letter only\.

Option A is the supporting answer and option B the misinformation answer iff the md5 hash of the query id is even—deterministic and computable from the query id alone\. In 44 of 149 rows the two supporting documents assert different valid interpretations; the menu gold then follows support1, frozen uniformly across readers\.

Scoring\.Answers are normalized \(lowercase; punctuation stripped; articles removed; whitespace collapsed\) and scored as the maximum over gold aliases, as exact match or token\-level F1\. The prediction is the last line matchinganswer:; failing that, the last nonempty line \(a leading “final answer:” is stripped\)\. Forced\-choice responses use a tiered parser: \(1\) exact single letter; \(2\) leading letter with separator and matching option text; \(3\) unambiguous option\-text mention; otherwise tier\-0, scored incorrect \(per\-reader tier\-0 rates 0\.0–2\.6%\)\.

Utility operators\.LOO arm:U\(m,q,d\)=F1\(full 8\-document context\)−F1\(context minusd\)U\(m,q,d\)=\\mathrm\{F1\}\(\\text\{full 8\-document context\}\)\-\\mathrm\{F1\}\(\\text\{context minus \}d\)\. Single\-document arms:U⁡\(m,q,d\)=score⁡\(document​d​only\)−score⁡\(closed\-book\)U\(m,q,d\)=\\mathrm\{score\}\(\\text\{document \}d\\text\{ only\}\)\-\\mathrm\{score\}\(\\text\{closed\-book\}\), with score==F1 \(open\-ended\) or accuracy \(forced choice\)\.

## Appendix CPer\-reader utility statistics \(LOO arm\)

Table 4:Per\-reader utility statistics on the internal LOO arm\(800 cells per reader: 100 queries×\\times8 documents\)\. Nonzero rate: fraction of cells withU≠0U\\neq 0\. Gain:F1⁡\(full context\)−F1⁡\(closed\-book\)\\mathrm\{F1\}\(\\text\{full context\}\)\-\\mathrm\{F1\}\(\\text\{closed\-book\}\), averaged over queries\.Pooling all 36 reader pairs: dual\-nonzero sign conflicts occur in 1,067 of 3,206 cells \(33\.3%; query\-cluster bootstrap 95% CI \[0\.283, 0\.377\]; 10,000 resamples over the 100 queries\); activity asymmetry in 8,348 of 11,554 pair–cell cases \(72\.3%\)\. The dual\-nonzero support of 3,206 is pinned as the checksum of the sparsity calibration \(Appendix[E](https://arxiv.org/html/2608.17781#A5)\)\.

Variance decomposition\.Treating the tensor as a balanced reader×\\timesquery×\\timesposition layout \(one observation per cell\), the exact sums\-of\-squares decomposition attributes variance as follows \(query\-cluster bootstrap 95% CI, 10,000 resamples\): reader main effect 0\.4% \[0\.3, 1\.4\]; query 6\.0% \[4\.2, 7\.9\]; position 2\.3% \[1\.1, 4\.5\]; reader×\\timesquery 29\.8% \[25\.7, 33\.0\]; reader×\\timesposition 0\.8% \[0\.8, 1\.6\]; query×\\timesposition 23\.7% \[19\.8, 27\.6\]; reader×\\timesquery×\\timesposition \(residual\) 37\.1% \[33\.0, 40\.6\]\. All terms involving the reader sum to 68\.0% \[63\.4, 72\.5\] in this algebraic decomposition\. Because every reader×\\timesquery×\\timesposition cell is observed once, the three\-way term cannot be separated from measurement error or other residual variation; the 68\.0% sum must be read with that qualification\. Permutation tests \(10,000 simulations; values shuffled across readers within each query–position column, destroying all reader structure\) reject the null for the reader main effect \(observed 0\.4% vs\. null median 0\.08%,p<10−4p<10^\{\-4\}\) and for reader×\\timesquery \(29\.8% vs\. 8\.4%,p<10−4p<10^\{\-4\}\), but not for reader×\\timesposition \(0\.8% vs\. 0\.6%,p=0\.08p=0\.08\)\. The clean result is therefore the large reader×\\timesquery interaction, not a uniform reader shift; the 68\.0% reader\-involving sum is a broader descriptive total\.

## Appendix DFull split distributions

All estimates use the frozen stratified split\-half protocol \(1,000 splits; seeds 20260815 internal/PRISM, 20260817 external arms\)\. Table[5](https://arxiv.org/html/2608.17781#A4.T5)reports the full split distributions underlying Figure[3](https://arxiv.org/html/2608.17781#S5.F3)and Table[1](https://arxiv.org/html/2608.17781#S5.T1); Table[6](https://arxiv.org/html/2608.17781#A4.T6)the per\-position breakdown underlying Figures[4](https://arxiv.org/html/2608.17781#S5.F4)and[5](https://arxiv.org/html/2608.17781#S5.F5)\. On the internal arm, 80\.6% of ordinal splits exceed 0\.5 while 92\.5% of signed splits fall below 0\.3 \(11\.5% negative\)\.

Table 5:Split\-half reliability distributions\(percentiles over 1,000 splits; 999 for forced ordinal\)\. NN: mean fraction of readers whose nearest neighbor in one half’s distance matrix is also nearest in the other half’s\. PairedΔ\\Deltarows are per\-split ordinal−\-signed differences;Δformat\\Delta\_\{\\mathrm\{format\}\}is the per\-split forced−\-open signed difference\.Table 6:Per\-position conditional signed stability\.Support: dual\-nonzero cells pooled over splits\. Forced mislead1 is undefined on 50 splits \(zero support in a half\)\.
## Appendix EStable\-world null calibrations

Internal arm \(sparsity calibration\)\.The stable world assumes a fixed per\-pair conflict propensitypi​jp\_\{ij\}across queries while preserving the real sparsity mask and each pair’s total conflict count\. The primary variant permutes observed conflict indicators within each reader pair \(conditional Monte Carlo test; 5,000 simulations, observed 0\.138 on 3,206 dual\-nonzero cells\)\.

Table 7:Stable\-world calibrations of the internal signed geometry\.The EB fit is Beta\(21\.0, 42\.1\) over the pooled conflict rate 0\.333, with between\-pair varianceτ2=0\.0035\\tau^\{2\}=0\.0035\(stable between\-pair SD 0\.059\)\.The calibration also bounds what reliability is*measurable*at our sample sizes: under the stable null, the expected split\-halfρ\\rhogrows from 0\.104 at 20 queries to 0\.191 \(40\), 0\.269 \(60\), 0\.373 \(100\), 0\.463 \(140\), and 0\.548 \(200\); 400 simulations per size\. The internal arm’s signed 0\.138 at 100 queries is far below its 0\.373 stable\-world expectation, while the ordinal 0\.599 exceeds it\.

PRISM artifact controls\(200 null worlds×\\times200 splits each\)\. Identity null \(document identities resampled, chosen\-list lengths preserved\): RBO 0\.314 \[0\.021, 0\.545\]; Jaccard 0\.314 \[0\.050, 0\.523\]—against the observed 0\.786\. Order null \(chosen sets preserved, order shuffled\): RBO 0\.703 \[0\.611, 0\.771\]; Jaccard 0\.742 \(invariant by construction\)\. Exact size matching: median aggregation degenerates \(equal\-length short lists quantize RBO to a single constant\); the mean\-aggregated size\-matched geometry is*more*stable than the raw one \(0\.862\)\.

External arms\.The primary signed calibration preserves the observed dual\-nonzero support and conflict count within every reader\-pair×\\timesevidence\-position stratum, then permutes the conflict indicators across queries inside that stratum \(2,000 simulations\)\. The pair\-only variant drops the position stratification; the ordinal control swaps the two support positions within reader–query cells \(300 simulations\)\.

Table 8:External\-arm null calibrations\.Signed nulls are one\-sidedp≤p\_\{\\leq\}; type\-shuffle nulls arep≥p\_\{\\geq\}\(does instance\-level ordering exceed type\-level structure?\)\. The K==4 design leaves only the two support positions swappable, so type\-shuffle tests instance\-level support ordering only\.
## Appendix FTest–retest and decoding\-noise bounds

Internal arm \(exact duplicate\)\.All 1,000 conditions of one local reader \(Qwen3\.5\-9B, deterministic decoding\) were accidentally run twice\. Of 800 utility cells, 122 are informative \(nonzero in either run\); the informative sign\-change rate isf=0\.123f=0\.123; when both runs are nonzero the sign holds in 107/109 \(0\.982\)\. Answer agreement 0\.960; utility magnitude Pearson 0\.977\. Under independent per\-cell flips, observed≈\\approxtrue×\(1−2f\)2=0\.569×\\times\(1\-2f\)^\{2\}=0\.569\\timestrue, capping true signed reliability at0\.138/0\.569=0\.2420\.138/0\.569=0\.242—below the stable null \(0\.363\) and far below ordinal \(0\.599\)\. The run\-vs\-run distance matrices correlate at 0\.786 with mean\|Δ​d\|=0\.020\|\\Delta d\|=0\.020\. Scope: one local reader; API readers ran deterministic decoding and were not re\-run, so a multi\-reader repetition remains the gold standard\.

External arms \(planned triplicate\)\.120 stratified queries \(60 per dataset\)×\\times3 total passes\. RAMDocs pools tof=0\.104f=0\.104\(546/5,257 informative cells; attenuation 0\.628; max compatible true signed 0\.550\): noise cannot bridge the ordinal–signed gap \(0\.833 vs\. 0\.345\), though it could in principle account for the small signed\-vs\-null shortfall \(null 0\.376\)\. RAGuard pools tof=0\.216f=0\.216\(649/3,004; attenuation 0\.323\), leaving the bound uninformative \(ceiling\>1\>1\)\. Per\-readerffspans 0\.015–0\.373 \(RAMDocs\) and 0\.000–0\.664 \(RAGuard\), highest for the two reasoning\-oriented endpoints \(K3, GPT\-5\.6\-Luna\), consistent with their less constrained decoding\. Since decoding noise attenuates observed stability, it works*against*—never for—the stable signed geometry we report on RAGuard\.

## Appendix GForced\-choice label strata

Table 9:Forced\-choice signed stability by label stratum\(median \[2\.5, 97\.5\] over 1,000 splits; mislead1 defined on 950\)\. Balanced==mean of the two stratum distances per split\. Noise1 has only 264 dual\-nonzero cells \(low power\)\.Two findings\. First, the aggregate forced\-choice elevation is not a label\-mixing artifact: balancing the strata changes overall signed stability from 0\.479 to 0\.482\. Second, the mislead1 geometry is measurable only in the gold==A stratum\. When gold==B \(the misleading answer sits at A\), readers flip to the misleading letter nearly uniformly; with conflict rates constant across pairs the stratum contributes no measurable geometry\. Uniform flipping is itself maximal signed agreement, so both strata indicate far more consistent signed behavior than the open\-ended 0\.104—the asymmetry marks*where*cross\-reader variance lives, not instability\. Closed\-book forced accuracy spans 0\.61–0\.87 across readers, which limits headroom for support\-side utility\.

## Appendix HTransfer experiment details

Design: for each of 9 target readers and 50 pre\-registered queries \(25 NQ, 25 HotpotQA\), every source reader’s measured utilities induce a tie\-aware preference set \(positive\-utility documents in retrieval order; rank\-based backfill only to pad\), evaluated on the target reader: 187 unique transfer cells per target, 4,050 source×\\timestarget×\\timesquery cells in total\.

Table 10:Nearest\-source vs\. random\-source transfer regret\(F1 with the target’s own selection minus F1 with the transferred selection, averaged over 50 queries\)\. Nearest source by behavior\-profile distance under the saturated probe bank \(Appendix[I](https://arxiv.org/html/2608.17781#A9)\)\. Negative regret means the transferred set beat the target’s own set—possible because the target’s own selection is itself sparse and backfilled\.Noise floor: 47\.7% of source pairs select identical document sets for a query \(mean Jaccard 0\.421 among the rest; 3\.74 distinct sets per query\); mean\|\|regret\|\|is 0\.035 F1 against a mean per\-cell standard error of 0\.037; 54\.4% of cells have zero cross\-source spread; selections average 1\.08 positive\-utility documents against 2\.77 backfilled\. Predictability: behavior\-profile distance vs\. symmetrized regretρ=0\.047\\rho=0\.047\(reader\-label permutationp=0\.901p=0\.901, 10,000 permutations\); in\-sample oracle utility distance vs\. regretρ=−0\.271\\rho=\-0\.271\(p=0\.264p=0\.264\); per\-target directional tests averageρ=0\.335\\rho=0\.335but pool to−0\.050\-0\.050\(56 cells\)\. Split\-half reliability of the regret matrix itself:−0\.281\-0\.281\[−0\.530\-0\.530, 0\.001\] over 200 splits\. The frozen decision rule \(ρ≥0\.30\\rho\\geq 0\.30,p<0\.05p<0\.05, every nearest source≥10%\\geq 10\\%regret reduction, nearest direction no worse on both datasets\) was not met; per\-dataset nearest−\-random regret is−0\.017\-0\.017\(HotpotQA\) and−0\.008\-0\.008\(NQ\)\.

## Appendix IBehavioral probe bank \(uninformative\)

An earlier stage of this project probed readers with a 48\-pair fictional behavior bank \(six dimensions\) intended to predict utility geometry from behavioral profiles\. On the finalized 9\-reader roster the bank saturates: 7 of 9 readers answer all pairs at ceiling \(the two newest API readers at EM=1\.0=1\.0\), leaving nonzero behavioral variation for only two local readers\. The behavior–geometry association on this roster isρ=0\.265\\rho=0\.265\(permutationp=0\.424p=0\.424\)—not estimable with a saturated instrument\. We report this as a measurement\-ceiling observation, not as evidence about the behavior–utility relationship: resolving modern readers requires a harder probe bank, which we did not build\. No main\-text claim depends on the probe bank\.

## Appendix JArtifact map

Table 11:Primary claims and their frozen artifacts\.