Evidence Lock Before Commitment: A Frozen Interface Degrades LLM-as-Judge Evaluation

# A Frozen Interface Degrades LLM-as-Judge Evaluation
Source: [https://arxiv.org/html/2608.05353](https://arxiv.org/html/2608.05353)
## Evidence Lock Before Commitment: A Frozen Interface Degrades LLM\-as\-Judge Evaluation

###### Abstract

LLM judges are often asked to extract criteria and evidence before choosing between candidate answers\. This workflow assumes that the intermediate record preserves the information needed for a later verdict\. For reasoning\-capable models, visible field order does not reveal internal decision order, so we test an observable alternative: persist the evidence in one call and make it the exclusive input to the next\. Across 24,000 judgments over HelpSteer3, FeedbackQA, and CoVal, we compare standard pairwise judging, structured one\-call judging, two\-call evidence locking, and three\-call pointwise locking with Claude Sonnet 4\.5 and GPT\-5\. Evidence locking reduces agreement with released human preferences by 4 to 6 percentage points and increases answer\-order inconsistency by 8 to 10 points relative to structured one\-call judging\. Pointwise locking is also harmful, while structured evidence elicitation remains close to standard judging\. The result holds for both judges and all three datasets\. Persisted evidence can support auditability, but it should not replace the source answers at decision time\.

Evidence Lock Before Commitment: A Frozen Interface Degrades LLM\-as\-Judge Evaluation

Divyansh SinghUniversity of Floridadivyansh\.singh@ufl\.edu

## 1Introduction

LLM\-as\-a\-judge evaluation now supports preference\-data construction, benchmark scoring, and model comparison at scale\(Zhenget al\.,[2023](https://arxiv.org/html/2608.05353#bib.bib1)\)\. Pairwise judging is simple to deploy, but its output depends on presentation order, superficial cues, and the elicitation protocol\(Shiet al\.,[2025](https://arxiv.org/html/2608.05353#bib.bib2); Chenet al\.,[2024](https://arxiv.org/html/2608.05353#bib.bib5); Huanget al\.,[2026](https://arxiv.org/html/2608.05353#bib.bib4)\)\. This has motivated evaluators that ask for criteria, plans, or evidence before a verdict\(Liuet al\.,[2023](https://arxiv.org/html/2608.05353#bib.bib12); Kimet al\.,[2024](https://arxiv.org/html/2608.05353#bib.bib13); Sahaet al\.,[2025](https://arxiv.org/html/2608.05353#bib.bib3)\)\.

These workflows make a stronger assumption than ordinary rationale prompting: that a natural\-language record contains enough information for a later call to reproduce the judgment\. In modular or auditable systems, that record may be persisted, reviewed, and passed between components\. Once it becomes the interface to the final decision, any omission or framing choice in the record becomes part of the evaluator itself\.

For reasoning\-capable models, visible output order does not reveal internal decision order\. A verdict written after an explanation does not establish that the model decided after reasoning\. We therefore intervene on observable information flow\. InEvidence Lock, one call inspects the context and both answers, writes evidence without selecting a winner, and persists the record\. A second call sees only that record and returns A, B, or tie\. InPointwise Lock, the answers are evaluated separately before a third call compares the two frozen assessments\. Standard and structured one\-call judging retain direct access to the source answers when producing the verdict\.

Across two judges, three datasets, and both answer orders, the locked protocols reduce preference agreement and increase order inconsistency\. The key comparison is evidence lock versus structured one\-call judging: both require explicit criteria and evidence, but only the lock freezes the artifact and removes the original answers from the final call\. The degradation therefore appears when the record becomes an exclusive interface, not when evidence is merely requested\.

Concretely, our contributions are: \(i\) anobservable evidence\-lock interventionthat does not rely on claims about hidden chain\-of\-thought order; \(ii\) a24,000\-judgment repeated\-measures evaluationacross Claude Sonnet 4\.5, GPT\-5, HelpSteer3, FeedbackQA, and CoVal; and \(iii\) aconsistent negative resultshowing that locked evidence reduces preference agreement and order robustness, while structured one\-call elicitation remains close to standard judging\.

## 2Related Work

Prior work motivates structured LLM evaluation but does not test when a persisted evidence record becomes the exclusive interface to a later verdict\. This is the gap we address\.

#### LLM judges and robustness\.

MT\-Bench and Chatbot Arena established LLM\-based pairwise evaluation while documenting position, verbosity, and self\-enhancement effects\(Zhenget al\.,[2023](https://arxiv.org/html/2608.05353#bib.bib1)\)\. Later work studies position bias, answer\-order swaps, and calibration against human preferences\(Shiet al\.,[2025](https://arxiv.org/html/2608.05353#bib.bib2); Liuet al\.,[2024](https://arxiv.org/html/2608.05353#bib.bib6)\)\. We use the same counterfactual swap, but ask whether the judging workflow changes robustness under that swap\.

#### Structured evaluation\.

G\-Eval combines explicit reasoning with form\-based scoring, and Prometheus evaluates responses against fine\-grained criteria\(Liuet al\.,[2023](https://arxiv.org/html/2608.05353#bib.bib12); Kimet al\.,[2024](https://arxiv.org/html/2608.05353#bib.bib13)\)\. EvalPlanner and PlanJudge place an explicit plan before the final evaluation\(Sahaet al\.,[2025](https://arxiv.org/html/2608.05353#bib.bib3); Huanget al\.,[2026](https://arxiv.org/html/2608.05353#bib.bib4)\)\. These methods motivate structured artifacts\. Our question is different: what happens when the artifact is frozen and the final judge can no longer inspect the original answers?

#### Rationales and faithfulness\.

Visible explanations need not faithfully reveal why a model produced its answer\(Turpinet al\.,[2023](https://arxiv.org/html/2608.05353#bib.bib14)\)\. LLM and human evaluators also exhibit systematic judgment biases\(Chenet al\.,[2024](https://arxiv.org/html/2608.05353#bib.bib5)\)\. We therefore define the intervention through call boundaries and source access, and treat rationale\-reference alignment as exploratory rather than as evidence of faithful internal reasoning\.

## 3Methodology

### 3\.1Observable judging protocols

Each input contains a task context and two candidate answers\. We compare four protocols \(Figure[1](https://arxiv.org/html/2608.05353#S3.F1)\)\. Every final decision allows A, B, or tie; only the information available to the decision call changes\.

![Refer to caption](https://arxiv.org/html/2608.05353v1/x1.png)Figure 1:Observable information flow in the four judging protocols\. Standard and structured judging retain source access at verdict time; locked protocols replace the source pair with persisted records\.Standard and structured judging differ in output format but not source access\. Evidence lock adds a persisted intermediate record and a second call that cannot see the original context or answers\. Pointwise lock further removes direct comparison during extraction by evaluating A and B independently\.

### 3\.2Outcomes and paired analysis

The statistical unit is the comparison ID\. Preference agreement is correctness against the released strict A/B label, averaged over AB and BA display orders within each protocol\. Answer\-order inconsistency is one binary outcome per comparison and protocol: whether the normalized verdict changes after the candidates are swapped\.

We report four paired contrasts: structured minus standard, evidence lock minus standard, pointwise lock minus standard, and evidence lock minus structured\. Confidence intervals use 10,000 nonparametric bootstrap resamples over comparison IDs\. Main\-text results emphasize point estimates; complete paired intervals are in Table[2](https://arxiv.org/html/2608.05353#Sx2.T2)\.

## 4Experimental Setup

### 4\.1Datasets and evaluation units

The three sources begin with different annotation structures\. We normalize each into the same pairwise unit:

> context \+ Answer A \+ Answer B \+ strict human\-preferred answer \+ human rationale\.

Table 1:Conversion of the released sources into a common pairwise evaluation unit\.HelpSteer3 already supplies a response pair\(Wanget al\.,[2025](https://arxiv.org/html/2608.05353#bib.bib7)\); FeedbackQA is converted by pairing the highest\- and lowest\-rated passages for each question\(Liet al\.,[2022](https://arxiv.org/html/2608.05353#bib.bib8)\); and CoVal contributes one strict\-majority pair per prompt\(OpenAI,[2025a](https://arxiv.org/html/2608.05353#bib.bib9)\)\. Every normalized unit contains a comparison ID, context, canonical Answers A and B, a strict gold label, released human rationales, and source metadata\. The sources support the same prediction task but need not have identical difficulty or definitions of quality, so we also report results by dataset\.

### 4\.2Sampling, judges, and generation

Using seed 20260803, we sample 500 units from each source, yielding 1,500 independent comparisons\. Every unit is evaluated under four protocols, both AB and BA orders, and two judges, yielding 24,000 valid judgments\. Verdicts from BA are mapped back to the canonical A/B labels before analysis\.

We use two reasoning\-capable proprietary judges served through their vendor APIs: Claude Sonnet 4\.5 \(claude\-sonnet\-4\-5\-20250929\)\(Anthropic,[2025](https://arxiv.org/html/2608.05353#bib.bib11)\)and GPT\-5 in its reasoning configuration \(gpt\-5\)\(OpenAI,[2025b](https://arxiv.org/html/2608.05353#bib.bib10)\)\. Both run with reasoning enabled and with the same protocol templates across datasets\. GPT\-5 runs at reasoning efforthigh; Claude Sonnet 4\.5 predates Anthropic’seffortparameter and uses a fixed extended\-thinking budget instead\. Neither vendor accepts temperature 0 with reasoning enabled, so decoding is not greedy and repeated calls need not be identical\. Full prompt templates, normalized records, and run metadata are in the appendices\.

### 4\.3Exploratory rationale analysis

For Claude Sonnet 4\.5, a separate model annotation pass compares judge explanations with the released rationale references\. It labels decisive\-criterion recall, unsupported claims, contradiction, shortcut use, alignment between verdict and rationale, and right\-for\-the\-wrong\-reason cases\. These are model\-derived labels against human references, not a human validation study; we use them only as secondary evidence\.

## 5Results

We ask three questions\. First, does a locked interface preserve agreement with released preferences? Second, does it improve robustness to answer order? Third, does the effect transfer across data sources? Figure[2](https://arxiv.org/html/2608.05353#S5.F2)reports pooled outcomes; paired confidence intervals are in Table[2](https://arxiv.org/html/2608.05353#Sx2.T2)\.

![Refer to caption](https://arxiv.org/html/2608.05353v1/x2.png)Figure 2:Pooled preference agreement and answer\-order inconsistency by protocol\. Structured one\-call judging remains close to standard judging, whereas both locked protocols reduce agreement and increase inconsistency\.### 5\.1Evidence locking reduces preference agreement

Structured judging remains close to the standard protocol for both judges: 76\.0% versus 75\.3% for Claude Sonnet 4\.5, and 75\.6% versus 75\.8% for GPT\-5\. Evidence lock falls to 71\.7% and 70\.0%, while pointwise lock falls to 67\.3% and 69\.2%\. Relative to structured judging, evidence lock reduces agreement by 4\.3 and 5\.6 percentage points\. Explicit evidence elicitation is therefore not the source of the loss; the drop appears when the record becomes frozen and exclusive\.

### 5\.2Locked protocols increase answer\-order inconsistency

Evidence lock raises inconsistency from 10\.9% to 18\.9% for Claude Sonnet 4\.5 and from 8\.3% to 15\.9% for GPT\-5\. Relative to structured judging, these are increases of 9\.5 and 8\.0 points\. Pointwise lock also remains substantially less robust than standard judging\. Separating the answers during extraction therefore does not remove the order sensitivity of the final decision\.

### 5\.3The effect transfers across datasets

Evidence lock is less accurate than standard judging in all six judge and dataset combinations\. The changes are−2\.4\-2\.4,−4\.5\-4\.5, and−3\.8​pp\-3\.8\\,\\mathrm\{pp\}for Claude Sonnet 4\.5 on HelpSteer3, FeedbackQA, and CoVal, and−4\.2\-4\.2,−5\.9\-5\.9, and−7\.2​pp\-7\.2\\,\\mathrm\{pp\}for GPT\-5\. Pointwise locking is also negative in all six cells\. The result is not driven by one source or one definition of quality\.

### 5\.4Locking costs more computation

For Claude Sonnet 4\.5, mean completion tokens per displayed\-order judgment rise from 1,279 under standard judging and 1,864 under structured judging to 4,183 under evidence lock and 6,825 under pointwise lock\. The four protocols require one, one, two, and three calls, respectively\. Locked judging therefore uses more calls and tokens while producing lower agreement and worse order robustness\.

The exploratory rationale pass moves in the same direction: locked protocols recover the decisive human criterion less often and produce more right\-for\-the\-wrong\-reason cases\. Full results and intervals are in Appendix[C](https://arxiv.org/html/2608.05353#A3)\.

## 6Discussion

#### The evidence record is not a sufficient interface\.

Structured and locked judging both require explicit criteria and evidence, yet only the locked protocols degrade sharply\. The natural\-language record can omit distinctions, compress tradeoffs, or preserve position\-dependent framing\. Once the final call loses source access, it cannot repair those choices\.

#### Auditability should preserve source access\.

Persisted evidence can still support review, logging, and modularity\. The practical lesson is narrower: the record should assist the decision rather than replace the original pair\. A final judge can inspect both the artifact and the source answers, and systems can allow the artifact to be corrected when it is incomplete\.

#### The result is an information\-flow effect, not a hidden\-reasoning claim\.

The experiment identifies what the final call can observe, not when the model internally decides\. The intervention also bundles stage separation, an additional call, record freezing, and source\-answer removal\. The information\-bottleneck account is therefore a plausible interpretation rather than a uniquely identified mechanism\.

## 7Conclusion

Evidence before commitment is not automatically beneficial\. Across two judges and three datasets, making a persisted evidence record the exclusive input to the verdict reduces preference agreement, increases answer\-order inconsistency, and costs more computation; structured one\-call evidence elicitation does not show the same degradation\.

## Limitations

The evaluation covers two proprietary reasoning\-capable judges under one fixed decoding configuration\. Other model families, providers, and settings may respond differently, and a hosted endpoint can change beneath a fixed model identifier\.

Evidence lock is a bundled intervention\. It adds a call, freezes the record, separates extraction from decision, and removes source access from the final call\. The design establishes the effect of that workflow but does not isolate which component causes the degradation\.

The targets are released human preferences rather than objective correctness labels\. They may contain annotation noise and source\-specific notions of quality\. The rationale analysis is also model\-derived and should not be read as human\-validated explanation quality\.

Each dataset contributes 500 sampled comparisons\. This supports paired estimation across protocols and orders, but it does not exhaust the full range of tasks or annotation conditions in any source\.

## Ethical Considerations

The study uses released text and preference annotations and collects no new participant data\. We retain only fields needed for evaluation and follow the source licenses\. Automated judges should not replace domain experts in high\-stakes decisions\. The anonymous review artifact includes prompts, seeded comparison IDs, both answer orders, stage records, and comparison\-level bootstrap code\.

Table 2:Paired contrast intervals by judge\. Percentage\-point changes with 95% bootstrap CIs in brackets\.
## References

- Anthropic \(2025\)Claude Sonnet 4\.5 system card\.Note:Anthropic publicationExternal Links:[Link](https://www.anthropic.com/news/claude-sonnet-4-5)Cited by:[§4\.2](https://arxiv.org/html/2608.05353#S4.SS2.p2.1)\.
- G\. H\. Chen, S\. Chen, Z\. Liu, F\. Jiang, and B\. Wang \(2024\)Humans or LLMs as the judge? a study on judgement bias\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 8301–8327\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.474),[Link](https://aclanthology.org/2024.emnlp-main.474/)Cited by:[§1](https://arxiv.org/html/2608.05353#S1.p1.1),[§2](https://arxiv.org/html/2608.05353#S2.SS0.SSS0.Px3.p1.1)\.
- H\. Huang, X\. Wu, M\. Yang, and Y\. Arase \(2026\)Reasoning model is superior LLM\-judge, yet suffers from biases\.arXiv preprint arXiv:2601\.03630\.External Links:[Link](https://arxiv.org/abs/2601.03630)Cited by:[§1](https://arxiv.org/html/2608.05353#S1.p1.1),[§2](https://arxiv.org/html/2608.05353#S2.SS0.SSS0.Px2.p1.1)\.
- S\. Kim, J\. Shin, Y\. Cho, J\. Jang, S\. Longpre, H\. Lee, S\. Yun, S\. Shin, S\. Kim, J\. Thorne, and M\. Seo \(2024\)Prometheus: inducing fine\-grained evaluation capability in language models\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=8euJaTveKw)Cited by:[§1](https://arxiv.org/html/2608.05353#S1.p1.1),[§2](https://arxiv.org/html/2608.05353#S2.SS0.SSS0.Px2.p1.1)\.
- Z\. Li, P\. Sharma, X\. H\. Lu, J\. C\. K\. Cheung, and S\. Reddy \(2022\)Using interactive feedback to improve the accuracy and explainability of question answering systems post\-deployment\.arXiv preprint arXiv:2204\.03025\.External Links:[Link](https://arxiv.org/abs/2204.03025)Cited by:[§4\.1](https://arxiv.org/html/2608.05353#S4.SS1.p3.1)\.
- Y\. Liu, D\. Iter, Y\. Xu, S\. Wang, R\. Xu, and C\. Zhu \(2023\)G\-eval: NLG evaluation using GPT\-4 with better human alignment\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Singapore,pp\. 2511–2522\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.153),[Link](https://aclanthology.org/2023.emnlp-main.153/)Cited by:[§1](https://arxiv.org/html/2608.05353#S1.p1.1),[§2](https://arxiv.org/html/2608.05353#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Liu, H\. Zhou, Z\. Guo, E\. Shareghi, I\. Vulić, A\. Korhonen, and N\. Collier \(2024\)Aligning with human judgement: the role of pairwise preference in large language model evaluators\.Proceedings of the Conference on Language Modeling\.External Links:[Link](https://arxiv.org/abs/2403.16950)Cited by:[§2](https://arxiv.org/html/2608.05353#S2.SS0.SSS0.Px1.p1.1)\.
- OpenAI \(2025a\)CoVal: a dataset of human preference comparisons and rationales\.Note:Hugging Face datasetExternal Links:[Link](https://huggingface.co/datasets/openai/coval)Cited by:[§4\.1](https://arxiv.org/html/2608.05353#S4.SS1.p3.1)\.
- OpenAI \(2025b\)GPT\-5 system card\.Note:OpenAI publicationExternal Links:[Link](https://openai.com/index/gpt-5-system-card/)Cited by:[§4\.2](https://arxiv.org/html/2608.05353#S4.SS2.p2.1)\.
- S\. Saha, X\. Li, M\. Ghazvininejad, J\. Weston, and T\. Wang \(2025\)Learning to plan & reason for evaluation with thinking\-LLM\-as\-a\-judge\.arXiv preprint arXiv:2501\.18099\.External Links:[Link](https://arxiv.org/abs/2501.18099)Cited by:[§1](https://arxiv.org/html/2608.05353#S1.p1.1),[§2](https://arxiv.org/html/2608.05353#S2.SS0.SSS0.Px2.p1.1)\.
- L\. Shi, C\. Ma, W\. Liang, X\. Diao, W\. Ma, and S\. Vosoughi \(2025\)Judging the judges: a systematic study of position bias in LLM\-as\-a\-judge\.InProceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics,pp\. 292–314\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.ijcnlp-long.18),[Link](https://aclanthology.org/2025.ijcnlp-long.18/)Cited by:[§1](https://arxiv.org/html/2608.05353#S1.p1.1),[§2](https://arxiv.org/html/2608.05353#S2.SS0.SSS0.Px1.p1.1)\.
- M\. Turpin, J\. Michael, E\. Perez, and S\. R\. Bowman \(2023\)Language models don’t always say what they think: unfaithful explanations in chain\-of\-thought prompting\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Document](https://dx.doi.org/10.52202/075280-3275),[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/ed3fea9033a80fea1376299fa7863f4a-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2608.05353#S2.SS0.SSS0.Px3.p1.1)\.
- Z\. Wang, J\. Zeng, O\. Delalleau, H\. Shin, F\. Soares, A\. Bukharin, E\. Evans, Y\. Dong, and O\. Kuchaiev \(2025\)HelpSteer3\-preference: open human\-annotated preference data across diverse tasks and languages\.arXiv preprint arXiv:2505\.11475\.External Links:[Link](https://arxiv.org/abs/2505.11475)Cited by:[§4\.1](https://arxiv.org/html/2608.05353#S4.SS1.p3.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2023\)Judging LLM\-as\-a\-judge with MT\-bench and chatbot arena\.arXiv preprint arXiv:2306\.05685\.External Links:[Link](https://arxiv.org/abs/2306.05685)Cited by:[§1](https://arxiv.org/html/2608.05353#S1.p1.1),[§2](https://arxiv.org/html/2608.05353#S2.SS0.SSS0.Px1.p1.1)\.

## Appendix AProtocol Templates

#### Standard pairwise\.

The judge receives the task context and both answers: “Compare the answers\. Explain your reasoning and select A, B, or tie\.” It returns analysis, decisive criterion, and verdict fields\.

#### Structured pairwise\.

The instruction is: “Identify criteria and exact evidence from both answers, compare them using the decisive criterion, then select A, B, or tie\.” The call returns evidence for A and B, criteria, decisive criterion, comparison, and verdict fields\.

#### Evidence lock, extraction\.

“Identify relevant strengths, errors, and evidence in A and B\. Do not select a winner\. Do not state which answer is better\. Return only structured evidence and applicable criteria\.” The parsed object is serialized and hashed\.

#### Evidence lock, decision\.

“Using only the locked evidence and criteria below, select A, B, or tie\.” The original context and answers are absent\.

#### Pointwise lock\.

A and B are evaluated in separate calls without access to the other answer\. A third call sees only the two serialized assessments and chooses A, B, or tie\.

## Appendix BNormalized Dataset Schema and Sampling

After source\-specific conversion, each sampled unit follows this record structure:

```
comparison ID
prompt or conversation context
canonical Answer A
canonical Answer B
gold label: A or B
released human rationales
source metadata
```

Every unit is evaluated in both AB and BA display orders\. Before computing agreement or order inconsistency, verdicts from BA are remapped to the canonical labels\. Canonical winner labels are balanced within each source\. The supplementary experiment record contains the seeded sample and data\-quality manifest\.

## Appendix CExploratory Rationale Outcomes

Table 3:Rationale outcomes \(%\) for Claude Sonnet 4\.5\.Relative to standard judging, evidence lock reduces decisive\-criterion recall by8\.2​pp8\.2\\,\\mathrm\{pp\}\(95% CI\[−10\.0,−6\.4\]\[\-10\.0,\-6\.4\]\), pointwise lock reduces it by11\.3​pp11\.3\\,\\mathrm\{pp\}\(\[−13\.3,−9\.3\]\[\-13\.3,\-9\.3\]\), and evidence lock increases right\-for\-the\-wrong\-reason \(RWFR\) cases by1\.7​pp1\.7\\,\\mathrm\{pp\}\(\[\+0\.7,\+2\.7\]\[\+0\.7,\+2\.7\]\)\. The evidence\-lock change in unsupported claims is inconclusive\.

## Appendix DJudge Run Configuration

The judges are Claude Sonnet 4\.5 \(claude\-sonnet\-4\-5\-20250929\), with extended thinking at a fixedbudget\_tokensbudget, and GPT\-5 \(gpt\-5\), at reasoning efforthigh\. Neither API accepts temperature 0 in that mode, so decoding is not greedy\. Remaining endpoint and decoding metadata are inRUN\_CONFIGURATION\.md\.

## Appendix EDataset Descriptives

Median answer\-pair character counts are 3,128 for HelpSteer3, 1,637 for FeedbackQA, and 1,042 for CoVal; the 95th percentiles are 7,629, 5,494, and 1,329\. Median counts of released human rationale or feedback statements are 3, 4, and 14, respectively\. No sampled comparison has an empty rationale set\.