Equitable System-Prompt Selection via Constrained Mixed-Strategy GroupDRO

# Equitable System-Prompt Selection via Constrained Mixed-Strategy GroupDRO
Source: [https://arxiv.org/html/2608.04339](https://arxiv.org/html/2608.04339)
Mengyu Xu1, Qiaoxin Yang2, Zhihan Liu3, Ruiyao Xu3, Zachary Liu4, Kezhen Chen5, Chongyang Gao3\\corresponding

###### Abstract

Large language models are increasingly used for information seeking, yet semantically equivalent questions phrased in different ways can receive answers of considerably different quality\. System prompts are widely employed to steer response behavior, but they are typically optimized for average\-case quality, so some question phrasings may still receive incomplete or low\-quality answers\. To address this, we formulate a constrained mixed\-strategy GroupDRO framework for system\-prompt selection\. Instead of optimizing the system\-prompt text, the framework assigns weights to system prompts in an existing pool to minimize the worst\-case information\-quality loss across evaluation metrics and groups, while constraining the mean loss to stay close to that of average\-based selection\. Because pool generation and selection are decoupled, the method applies to any system\-prompt pool and can leverage an ensemble of complementary system prompts rather than a single one\. Across five LLMs on two bilingual medical and consumer\-finance benchmarks, the constrained method reduces the Overall Mean, Worst 25% Mean, and Worst by 13\.1%, 13\.2%, and 13\.7% on average relative to no mitigation while keeping overall quality close to Average selection\. Its multi\-prompt weights reveal complementarity across metric\-group pairs\. Code and data are available at[https://github\.com/Rainxu09/equitable\-system\-prompt\-selection](https://github.com/Rainxu09/equitable-system-prompt-selection)\.

## 1Introduction

Large language models \(LLMs\) are increasingly used for everyday information seeking in high\-stakes domains such as health and personal finance\. In these settings, the same underlying question can be phrased in many ways\. Ideally, semantically equivalent questions should receive answers of comparable quality\. In practice, however, response quality can vary considerably with phrasing, and prior work shows that LLMs can respond differently to content\-equivalent inputs depending on language, domain literacy, resource constraints, dialect, persona, or framing signals\(Xuet al\.[2026](https://arxiv.org/html/2608.04339#bib.bib5); Hofmannet al\.[2024](https://arxiv.org/html/2608.04339#bib.bib10); Guptaet al\.[2024](https://arxiv.org/html/2608.04339#bib.bib12)\)\. This disparity has a fairness dimension: users whose phrasing signals low domain literacy are precisely those least able to recover missing or diluted information on their own, and low health literacy is linked to poorer health outcomes\(Berkmanet al\.[2011](https://arxiv.org/html/2608.04339#bib.bib11)\)\.

A common mitigation is to add a system prompt that steers response behavior, for example by instructing the model to preserve key information or maintain actionable detail\. However, whether written by hand or generated by automated prompt optimizers\(Zhouet al\.[2023](https://arxiv.org/html/2608.04339#bib.bib7); Yanget al\.[2024](https://arxiv.org/html/2608.04339#bib.bib13); Guoet al\.[2024](https://arxiv.org/html/2608.04339#bib.bib24); Agrawalet al\.[2026](https://arxiv.org/html/2608.04339#bib.bib15)\), system prompts are typically selected for their average response quality\. A system prompt that performs well on average can still leave particular question phrasings with incomplete or low\-quality answers, and average\-based selection provides no mechanism to detect or correct this\.

We address this problem at the selection stage\. Strong candidate system prompts are already abundant, coming from human design, LLM generation, or automatic prompt optimizers, so the practical question is not how to write another prompt but how to use the existing ones well\. Furthermore, no single system prompt may serve all question phrasings well, so a method that combines complementary prompts offers protection that any single rewritten prompt cannot\. We formulate the problem in a practical regime: given a finite pool of candidate system prompts, predefined evaluation groups, and offline scores for each candidate, we ask how to select from the pool so that the weakest groups also receive high\-quality answers\. These conditions make the problem exactly solvable rather than a learning task\. With a finite pool and fully observed scores, there is no unseen input space to generalize over; a fitted selector would only add estimation error, which could itself fall unevenly across groups\.

Building on these considerations, we propose a constrained mixed\-strategy GroupDRO method for system\-prompt selection, which carries the worst\-group principle of group distributionally robust optimization\(Sagawaet al\.[2020](https://arxiv.org/html/2608.04339#bib.bib3)\)and minimax fairness\(Hashimotoet al\.[2018](https://arxiv.org/html/2608.04339#bib.bib6); Dianaet al\.[2021](https://arxiv.org/html/2608.04339#bib.bib4)\)from model training to prompt selection\. The selector assigns weights to system prompts so as to minimize the worst\-case information\-quality loss across evaluation metrics and groups, while a mean\-loss constraint keeps average quality close to that of average\-based selection\. Because the objective is linear in the weights, the problem reduces to a linear program: model agnostic, free of additional training, and applicable to any prompt pool\. Beyond selection, the optimal weights serve as a diagnostic\. If the solution concentrates on a single system prompt, that prompt alone is sufficient under the worst\-case objective; if it spreads weight across several, the pool contains complementary prompts that protect different evaluation metrics and groups\.

We evaluate this approach across five LLMs on two bilingual information\-seeking benchmarks: MIRA, an existing medical information benchmark\(Xuet al\.[2026](https://arxiv.org/html/2608.04339#bib.bib5)\), and a new controlled consumer finance benchmark that we construct\. In our experiments, the constrained method improves the overall mean score, the mean score of the worst 25% of metric–group pairs, and the worst\-case score relative to the no\-mitigation baseline, reducing them by 13\.1%, 13\.2%, and 13\.7% on average, respectively\. It achieves consistent gains over single\-prompt selectors on the weakest quartile in all 10 model–domain settings, while keeping overall average quality nearly unchanged\. The selected weights consistently place mass on multiple system prompts, showing that different system prompts protect different evaluation metrics and groups\. Our contributions can be summarized as follows\.

- •An equitable system\-prompt selection framework with a constrained mixed\-strategy GroupDRO selector\. We formulate mitigation prompting as selection from an existing candidate pool rather than optimization of prompt text: the selector assigns weights to system prompts to minimize the worst\-case information\-quality loss across evaluation metrics and groups, subject to a mean\-loss constraint that preserves average quality\. The method is model agnostic, requires no additional training, and applies to pools from any source\. The optimal weights also serve as a diagnostic of prompt complementarity in the pool\.
- •A bilingual consumer\-finance benchmark\. We construct a controlled bilingual consumer\-finance benchmark with 60 low\-risk seed questions across six categories and fixed information checklists across question phrasings\. Finance experts reviewed the seeds, checklists, and rubrics, and we audit agreement between a trained finance annotator and the LLM judge on 250 responses\.
- •A cross\-domain empirical study\. Across bilingual medical and consumer\-finance benchmarks and five LLMs, we show that constrained mixed\-strategy GroupDRO system\-prompt selection improves answer quality for the weakest metric–group pairs compared with the no\-mitigation baseline and single\-prompt selectors, while keeping average quality nearly unchanged\.

## 2Background and Related Work

##### Group\-robust optimization and minimax fairness\.

GroupDRO and minimax fairness focus on the worst\-performing group, not only the average case\(Sagawaet al\.[2020](https://arxiv.org/html/2608.04339#bib.bib3); Hashimotoet al\.[2018](https://arxiv.org/html/2608.04339#bib.bib6); Dianaet al\.[2021](https://arxiv.org/html/2608.04339#bib.bib4)\)\. Most prior work uses this idea during training or fine\-tuning, where the method updates model weights and evaluates groups defined in the training data\. We apply the same worst\-group idea to system\-prompt selection\. Without changing the model or rewriting the system prompt, we select from an existing pool to reduce worst\-group information\-quality loss\.

##### Prompt optimization and system\-prompt selection\.

A broader review of prompt optimization and system\-prompt selection is provided in Appendix[A](https://arxiv.org/html/2608.04339#A1)\. Unlike prompt optimization methods that generate or modify prompt text, our method operates on an existing system\-prompt pool\. Among existing selection approaches, the closest to our setting is Prompt Risk Control \(PRC\)\(Zolloet al\.[2024](https://arxiv.org/html/2608.04339#bib.bib9)\), which uses calibration data to select a prompt whose risk satisfies a chosen bound with high probability\. We include PRC as a baseline in our experiments\.

Prompt optimization and selection methods generally seek a single prompt that performs well under a distribution\-level criterion, such as average quality or a tail\-risk bound\. Such criteria do not identify or optimize the weakest evaluation groups: a prompt that is strong on average, or whose response\-level tail is controlled, can still leave particular groups with weaker answers\. Given an existing system\-prompt pool, we instead select a system prompt, or a weighted mixture, that protects the weakest metric–group pairs while preserving average quality\. To our knowledge, this is the first work to formulate system\-prompt selection as a constrained group\-robust optimization problem\.

##### LLM fairness and question phrasing effects\.

Our work also builds on fairness and bias evaluation in LLMs\. Prior work shows that LLMs can respond differently to content\-equivalent inputs depending on language, literacy, dialect, persona, or framing signals\(Xuet al\.[2026](https://arxiv.org/html/2608.04339#bib.bib5); Hofmannet al\.[2024](https://arxiv.org/html/2608.04339#bib.bib10); Guptaet al\.[2024](https://arxiv.org/html/2608.04339#bib.bib12)\)\. In medical and financial information seeking, low domain literacy is especially important because users may be less able to fill in missing or diluted information\. Prior work also links low health literacy to poorer health outcomes\(Berkmanet al\.[2011](https://arxiv.org/html/2608.04339#bib.bib11)\)\. We do not treat all groups as fairness groups\. Low health\- and financial\-literacy signals give the clearest fairness interpretation, while language, register, query skeleton, resource constraints, and expression framing mainly test robustness to different ways of asking the same question\.

## 3Methods

### 3\.1Problem formulation

##### Setup and notation\.

Let𝒫=\{p1,…,pN\}\\mathcal\{P\}=\\\{p\_\{1\},\\dots,p\_\{N\}\\\}denote a fixed pool ofNNcandidate system prompts\. For an application domain, let𝒢\\mathcal\{G\}denote its predefined evaluation groups\. The construction of𝒢\\mathcal\{G\}is domain\-specific and is described in Section[4\.2](https://arxiv.org/html/2608.04339#S4.SS2)\. Each group corresponds to a distinct profile of question characteristics applied to the same underlying question\. Depending on the domain, these characteristics include language, literacy signal, resource constraint, register, question skeleton, and framing\. Letℳ\\mathcal\{M\}denote the set of evaluation metrics\. In our experiments,ℳ=\{m1,m2,m3\}\\mathcal\{M\}=\\\{m\_\{1\},m\_\{2\},m\_\{3\}\\\}, wherem1m\_\{1\}measures information dilution,m2m\_\{2\}measures completeness, andm3m\_\{3\}measures actionability\. Information dilution\(Xuet al\.[2026](https://arxiv.org/html/2608.04339#bib.bib5)\)refers to responses that address the question but omit or weaken substantive information, such as underlying mechanisms, relevant thresholds, or risk boundaries\.

For system promptp∈𝒫p\\in\\mathcal\{P\}, evaluation groupg∈𝒢g\\in\\mathcal\{G\}, and evaluation metricm∈ℳm\\in\\mathcal\{M\}, letLp,gmL^\{m\}\_\{p,g\}denote the empirical information\-quality loss of system promptppon groupggfor metricmm\. Lower values indicate better responses\. We assume that all losses are nonnegative\. For each application domain, we construct a joint loss matrix\. Rows correspond to candidate system prompts, columns correspond to metric–group pairs inℳ×𝒢\\mathcal\{M\}\\times\\mathcal\{G\}, and each entry is the mean LLM\-judge score of responses generated for system promptpp, groupgg, and metricmm\. Because lower scores indicate better responses, our optimization treats these scores as losses to minimize\. The equations below optimize jointly over evaluation metrics and groups\.

##### Single\-Prompt Selection\.

A common approach is to select one system prompt from the fixed pool and use it across all evaluation groups\. Average\-loss selection selects one system prompt by minimizing mean loss across evaluation metrics and groups:

pavg=arg​minp∈𝒫⁡1\|ℳ\|​\|𝒢\|​∑m∈ℳ∑g∈𝒢Lp,gm\.p\_\{\\mathrm\{avg\}\}=\\operatorname\*\{arg\\,min\}\_\{p\\in\\mathcal\{P\}\}\\frac\{1\}\{\|\\mathcal\{M\}\|\|\\mathcal\{G\}\|\}\\sum\_\{m\\in\\mathcal\{M\}\}\\sum\_\{g\\in\\mathcal\{G\}\}L^\{m\}\_\{p,g\}\.\(1\)
However, a system prompt that performs well on average may still underperform on certain metric\-group pairs\. Pure GroupDRO\(Sagawaet al\.[2020](https://arxiv.org/html/2608.04339#bib.bib3)\)addresses this by selecting the system prompt with the lowest worst\-case loss:

ppure⋆=arg​minp∈𝒫⁡maxm∈ℳg∈𝒢⁡Lp,gm\.p^\{\\star\}\_\{\\mathrm\{pure\}\}=\\operatorname\*\{arg\\,min\}\_\{p\\in\\mathcal\{P\}\}\\max\_\{\\begin\{subarray\}\{c\}m\\in\\mathcal\{M\}\\\\ g\\in\\mathcal\{G\}\\end\{subarray\}\}L^\{m\}\_\{p,g\}\.\(2\)
Both methods rely on a single system prompt, which can be limiting when different system prompts help different metric\-group pairs\. Moreover, a single prompt may not balance the trade\-off between average quality and worst\-case protection: although Pure GroupDRO finds the best single prompt under the joint worst\-case objective, that prompt may still fail to provide comparable answer quality across all evaluation metrics and groups\. These limitations motivate the constrained mixed\-strategy formulation introduced next\.

### 3\.2Constrained Mixed\-strategy GroupDRO

Mixed strategies are a standard tool in minimax problems\(v\. Neumann[1928](https://arxiv.org/html/2608.04339#bib.bib1)\)and robust decision\-making\(Sessaet al\.[2020](https://arxiv.org/html/2608.04339#bib.bib2)\)\. We use this framework as a relaxation of single\-system\-prompt selection\. Mixed\-strategy GroupDRO uses the same fixed pool but assigns weights to system prompts, testing whether this weighted mixture can achieve lower worst\-case loss across metric–group pairs than the best single system prompt\. To prevent this worst\-case gain from degrading average quality, we further constrain the mean loss of the mixture to stay close to that of Average selection\. If several system prompts receive weight under this constraint, this indicates that different system prompts in the pool help different evaluation metrics or groups\. In other words, the pool contains complementary system prompts\.

##### Weighted system\-prompt mixture\.

Mixed GroupDRO chooses a distribution over system prompts, which we interpret as weights over the system\-prompt pool\. Letw=\(w1,…,wN\)∈ΔNw=\(w\_\{1\},\\dots,w\_\{N\}\)\\in\\Delta\_\{N\}denote these weights, whereΔN=\{w∈ℝ≥0N:∑i=1Nwi=1\}\\Delta\_\{N\}=\\\{w\\in\\mathbb\{R\}^\{N\}\_\{\\geq 0\}:\\sum\_\{i=1\}^\{N\}w\_\{i\}=1\\\}andwiw\_\{i\}is the weight assigned to system promptpip\_\{i\}\. The expected loss for metricmmand evaluation groupggunder weightswwis:

Rm,g​\(w\)=∑i=1Nwi​Lpi,gm\.R\_\{m,g\}\(w\)=\\sum\_\{i=1\}^\{N\}w\_\{i\}L^\{m\}\_\{p\_\{i\},g\}\.\(3\)
Mixed GroupDRO chooses the system\-prompt weights that minimize the worst\-case expected loss\.

##### Mean\-constrained mixture\.

To keep average quality comparable to Average selection, we constrain the mean loss of the weighted mixture\. Let

L¯​\(w\)=1\|ℳ\|​\|𝒢\|​∑m∈ℳ∑g∈𝒢Rm,g​\(w\)\\bar\{L\}\(w\)=\\frac\{1\}\{\|\\mathcal\{M\}\|\|\\mathcal\{G\}\|\}\\sum\_\{m\\in\\mathcal\{M\}\}\\sum\_\{g\\in\\mathcal\{G\}\}R\_\{m,g\}\(w\)\(4\)denote the mean loss under weightsww\. Constrained Mixed GroupDRO chooses the system\-prompt weights that minimize the worst\-case expected loss while limiting the mean loss:

wϵ⋆=arg⁡minw∈ΔN\\displaystyle w^\{\\star\}\_\{\\epsilon\}=\\arg\\min\_\{w\\in\\Delta\_\{N\}\}maxm∈ℳg∈𝒢⁡Rm,g​\(w\)\\displaystyle\\max\_\{\\begin\{subarray\}\{c\}m\\in\\mathcal\{M\}\\\\ g\\in\\mathcal\{G\}\\end\{subarray\}\}R\_\{m,g\}\(w\)\(5\)subject toL¯​\(w\)≤\(1\+ϵ\)​L¯avg,\\displaystyle\\bar\{L\}\(w\)\\leq\(1\+\\epsilon\)\\,\\bar\{L\}\_\{\\mathrm\{avg\}\},where

L¯avg=1\|ℳ\|​\|𝒢\|​∑m∈ℳ∑g∈𝒢Lpavg,gm\\bar\{L\}\_\{\\mathrm\{avg\}\}=\\frac\{1\}\{\|\\mathcal\{M\}\|\|\\mathcal\{G\}\|\}\\sum\_\{m\\in\\mathcal\{M\}\}\\sum\_\{g\\in\\mathcal\{G\}\}L^\{m\}\_\{p\_\{\\mathrm\{avg\}\},g\}\(6\)denotes the mean loss achieved bypavgp\_\{\\mathrm\{avg\}\}\. We assumeϵ≥0\\epsilon\\geq 0\. In our main experiments, we setϵ=0\.005\\epsilon=0\.005, so the development mean loss can be at most 0\.5% higher than that of Average selection\.

For later use, define the constrained mixed feasible set asΔN,ϵ=\{w∈ΔN:L¯​\(w\)≤\(1\+ϵ\)​L¯avg\}\.\\Delta\_\{N,\\epsilon\}=\\left\\\{w\\in\\Delta\_\{N\}:\\bar\{L\}\(w\)\\leq\(1\+\\epsilon\)\\bar\{L\}\_\{\\mathrm\{avg\}\}\\right\\\}\.

##### Linear program formulation\.

The Constrained Mixed GroupDRO problem in Eq\. \([5](https://arxiv.org/html/2608.04339#S3.E5)\) minimizes the maximum over finitely many metric–group pairs, and bothRm,g​\(w\)R\_\{m,g\}\(w\)andL¯​\(w\)\\bar\{L\}\(w\)are linear inww\. It can therefore be written as a linear program\. We introduce a variablettthat upper\-bounds the expected loss of every metric–group pair:

minw,t\\displaystyle\\min\_\{w,t\}t\\displaystyle t\(7\)s\.t\.Rm,g​\(w\)≤t,∀m∈ℳ,∀g∈𝒢,\\displaystyle R\_\{m,g\}\(w\)\\leq t,\\quad\\forall m\\in\\mathcal\{M\},\\ \\forall g\\in\\mathcal\{G\},L¯​\(w\)≤\(1\+ϵ\)​L¯avg,\\displaystyle\\bar\{L\}\(w\)\\leq\(1\+\\epsilon\)\\bar\{L\}\_\{\\mathrm\{avg\}\},∑i=1Nwi=1,\\displaystyle\\sum\_\{i=1\}^\{N\}w\_\{i\}=1,wi≥0,i=1,…,N\.\\displaystyle w\_\{i\}\\geq 0,\\quad i=1,\\dots,N\.The second constraint limits the mean loss relative to average selection\. Removing this constraint gives the unconstrained Mixed GroupDRO formulation\.

For a fixed feasibleww, the smallest feasible value ofttismaxm∈ℳg∈𝒢⁡Rm,g​\(w\),\\max\_\{\\begin\{subarray\}\{c\}m\\in\\mathcal\{M\}\\\\ g\\in\\mathcal\{G\}\\end\{subarray\}\}R\_\{m,g\}\(w\),so the linear program in Eq\. \([7](https://arxiv.org/html/2608.04339#S3.E7)\) is equivalent to the Constrained Mixed GroupDRO objective in Eq\. \([5](https://arxiv.org/html/2608.04339#S3.E5)\)\. In other words, minimizingttminimizes the worst\-case expected loss while satisfying the mean\-loss constraint\.

To compare single\-system\-prompt and mixed selection under the same constraint, letei∈ΔNe\_\{i\}\\in\\Delta\_\{N\}denote the one\-hot vector that places all weight on system promptpip\_\{i\}\. Define the feasible set of single system prompts as

𝒫ϵ=\{pi∈𝒫:L¯​\(ei\)≤\(1\+ϵ\)​L¯avg\}\.\\mathcal\{P\}\_\{\\epsilon\}=\\left\\\{p\_\{i\}\\in\\mathcal\{P\}:\\bar\{L\}\(e\_\{i\}\)\\leq\(1\+\\epsilon\)\\bar\{L\}\_\{\\mathrm\{avg\}\}\\right\\\}\.SinceRm,g​\(ei\)=Lpi,gmR\_\{m,g\}\(e\_\{i\}\)=L^\{m\}\_\{p\_\{i\},g\},L¯​\(ei\)\\bar\{L\}\(e\_\{i\}\)is exactly the mean loss of system promptpip\_\{i\}\. Forϵ≥0\\epsilon\\geq 0, this set is nonempty because the one\-hot vector corresponding topavgp\_\{\\mathrm\{avg\}\}has mean lossL¯avg≤\(1\+ϵ\)​L¯avg\\bar\{L\}\_\{\\mathrm\{avg\}\}\\leq\(1\+\\epsilon\)\\bar\{L\}\_\{\\mathrm\{avg\}\}, and thereforepavg∈𝒫ϵp\_\{\\mathrm\{avg\}\}\\in\\mathcal\{P\}\_\{\\epsilon\}\.

The constrained pure and mixed worst\-case values are

Vpure,ϵ=minpi∈𝒫ϵ⁡maxm∈ℳg∈𝒢⁡Rm,g​\(ei\)V\_\{\\mathrm\{pure\},\\epsilon\}=\\min\_\{p\_\{i\}\\in\\mathcal\{P\}\_\{\\epsilon\}\}\\max\_\{\\begin\{subarray\}\{c\}m\\in\\mathcal\{M\}\\\\ g\\in\\mathcal\{G\}\\end\{subarray\}\}R\_\{m,g\}\(e\_\{i\}\)and, using the constrained feasible setΔN,ϵ\\Delta\_\{N,\\epsilon\},

Vmix,ϵ=minw∈ΔNL¯​\(w\)≤\(1\+ϵ\)​L¯avg⁡maxm∈ℳg∈𝒢⁡Rm,g​\(w\)\.V\_\{\\mathrm\{mix\},\\epsilon\}=\\min\_\{\\begin\{subarray\}\{c\}w\\in\\Delta\_\{N\}\\\\ \\bar\{L\}\(w\)\\leq\(1\+\\epsilon\)\\bar\{L\}\_\{\\mathrm\{avg\}\}\\end\{subarray\}\}\\max\_\{\\begin\{subarray\}\{c\}m\\in\\mathcal\{M\}\\\\ g\\in\\mathcal\{G\}\\end\{subarray\}\}R\_\{m,g\}\(w\)\.Here,Vpure,ϵV\_\{\\mathrm\{pure\},\\epsilon\}is the best worst\-case value among single system prompts satisfying the mean\-loss constraint\. It is distinct from the Pure GroupDRO objective in Eq\. \([2](https://arxiv.org/html/2608.04339#S3.E2)\)\.

###### Proposition 1\(Constrained Mixed GroupDRO dominance\)\.

Forϵ≥0\\epsilon\\geq 0,Vmix,ϵ≤Vpure,ϵV\_\{\\mathrm\{mix\},\\epsilon\}\\leq V\_\{\\mathrm\{pure\},\\epsilon\}\. Equality holds if and only if there is an optimal constrained mixed solution that places all weight on a single prompt in𝒫ϵ\\mathcal\{P\}\_\{\\epsilon\}\. WhenVmix,ϵ<Vpure,ϵV\_\{\\mathrm\{mix\},\\epsilon\}<V\_\{\\mathrm\{pure\},\\epsilon\}, every optimal constrained mixed solution places positive weight on at least two system prompts\.

The proof follows because every mean\-feasible single system prompt corresponds to a one\-hot vector in the constrained mixed feasible setΔN,ϵ\\Delta\_\{N,\\epsilon\}\. Thus, feasible single\-prompt choices form a subset of the constrained mixed feasible set, and the mixed optimum cannot be worse than the best feasible single prompt\. Equality holds when a one\-hot vector attains the mixed optimum\. A full proof is provided in Appendix[B](https://arxiv.org/html/2608.04339#A2)\.

The inequality is a development\-matrix guarantee, not a test guarantee\. We therefore use Constrained Mixed GroupDRO to diagnose system\-prompt\-pool complementarity\. If an optimal constrained mixed solution places all weight on one system prompt, the best mean\-feasible single system prompt is already minimax\-optimal under the constrained objective\. IfVmix,ϵ<Vpure,ϵV\_\{\\mathrm\{mix\},\\epsilon\}<V\_\{\\mathrm\{pure\},\\epsilon\}, no mean\-feasible single system prompt achieves the same worst\-case development loss, providing evidence of system\-prompt complementarity\.

## 4Experiments

### 4\.1System\-prompt Pool Construction

Our method takes the candidate system\-prompt pool𝒫\\mathcal\{P\}as input rather than optimizing system\-prompt text directly\. It can be constructed from sources, including human\-written system prompts, LLM\-generated system prompts, prompt\-optimization methods, or combinations of these sources\.

In our experiments, we generate𝒫\\mathcal\{P\}with GPT\-5\.4\(OpenAI[2026b](https://arxiv.org/html/2608.04339#bib.bib28)\)using human\-specified mitigation goals, then manually review the system prompts before selection\. Each domain uses a fixed pool of 55 system prompts, organized into 11 mitigation families with 5 wording variants per family\. The families target broad mitigation goals such as preserving checklist information, avoiding over\-referral, maintaining actionable detail, and adapting explanations without diluting domain content\. Within each domain, the same system\-prompt pool is used across all evaluation metrics and groups for all four selectors: Average selection, Pure GroupDRO, Mixed GroupDRO, and Constrained Mixed GroupDRO\. This lets us test whether selecting from a general system\-prompt pool can improve worst\-case robustness without writing separate system prompts for different evaluation metrics or groups\.

### 4\.2Benchmarks and Evaluation Groups

We evaluate on MIRA, an existing bilingual medical\-information benchmark\(Xuet al\.[2026](https://arxiv.org/html/2608.04339#bib.bib5)\), and on a new consumer\-finance benchmark constructed with the same controlled evaluation design\. Our finance benchmark contains 60 low\-risk consumer\-finance seed questions, balanced across six categories: securities, trusts, insurance, consumer protection, credit and credit reporting, and deposits and payments, with 10 seeds per category\. For each seed, we define a checklist of the key information that a useful answer should include, and use the same checklist across all phrasings of that question\. We create 24 finance groups by crossing two languages \(English and Chinese\), two financial\-literacy signals \(high and low\), two resource constraints \(high and low\), and three expression frames \(direct, confusion, and misconception\), yielding 1,440 finance question variants\. Resource constraint indicates whether the user has high or low assets\. Expression frame shows direct, confused, or misconception\-based phrasing\.

The consumer\-finance responses were annotated by a trained finance expert\. Before annotation, the finance seeds, checklists, and scoring rubrics were additionally reviewed by domain experts, including a professor in finance and economics\. Appendix[C](https://arxiv.org/html/2608.04339#A3)reports agreement between the finance annotator and the LLM judge on 250 audited responses\. Exact agreement is the proportion of identical ratings, adjacent agreement is the proportion of ratings that differ by at most one point, and QWK denotes quadratic weighted kappa\. Agreement is consistently high across the three evaluation metrics, with exact agreement ranging from 0\.804 to 0\.924, adjacent agreement from 0\.964 to 0\.996, and QWK from 0\.832 to 0\.923\. Each domain contains 60 underlying questions, with 20 used for development and 40 held out for testing\. The split preserves coverage across categories, language, literacy signal, and question framing\. Responses are scored on three information\-quality metrics:m1m\_\{1\}\(information dilution\),m2m\_\{2\}\(completeness\), andm3m\_\{3\}\(actionability\), with lower scores indicating better responses\. All three metrics are used jointly for system\-prompt selection and evaluation\.

### 4\.3Experimental Protocol

We evaluate five LLMs: Qwen36Plus\(Yanget al\.[2025](https://arxiv.org/html/2608.04339#bib.bib26)\), DeepSeek\-V4Pro\(DeepSeek AI[2026](https://arxiv.org/html/2608.04339#bib.bib27)\), GLM5\(GLM\-5\-Team[2026](https://arxiv.org/html/2608.04339#bib.bib30)\), Gemma4\-31B\(Teamet al\.[2026](https://arxiv.org/html/2608.04339#bib.bib31)\), and Llama4Scout\(Meta AI[2025](https://arxiv.org/html/2608.04339#bib.bib32)\)\. For each domain, we evaluate 60 seed questions across 24 evaluation groups, yielding 1,440 question variants in total\. Each model is run on the 55\-system\-prompt pool and on a no\-mitigation baseline, producing 792,000 system\-prompt\-pool responses and 14,400 baseline responses across the two domains, for 806,400 model responses\. Responses are scored by GPT\-5\.4\-mini\(OpenAI[2026a](https://arxiv.org/html/2608.04339#bib.bib33)\)as the LLM judge\(Zhenget al\.[2023](https://arxiv.org/html/2608.04339#bib.bib29)\)using the MIRA scoring framework, adapted to consumer\-finance content while keeping the same score direction and metrics\. Baseline responses are scored in the same way but are not included in system\-prompt selection\. They are used only to measure information\-quality differences before mitigation\.

For each model and domain, we score every system prompt on all metric–group pairs in the development set\. This gives a joint development score matrix with system prompts as rows and the 72 metric–group pairs as columns\. We then apply four selectors to the same matrix\. Average selection minimizes the mean score across all pairs, Pure GroupDRO minimizes the worst\-case score, Mixed GroupDRO minimizes the worst\-case score using system\-prompt weights, and Constrained Mixed GroupDRO solves the LP in \([7](https://arxiv.org/html/2608.04339#S3.E7)\) with the mean\-loss constraint\. For Constrained Mixed GroupDRO, we compareϵ∈\{0,0\.005,0\.01,0\.02\}\\epsilon\\in\\\{0,0\.005,0\.01,0\.02\\\}using the development\-set tradeoff between the Overall mean and the Worst 25% mean\. We useϵ=0\.005\\epsilon=0\.005in the main experiments because it lies on the development Pareto frontier in both domains\.

As an additional single\-system\-prompt baseline, we evaluate Prompt Risk Control \(PRC\)\(Zolloet al\.[2024](https://arxiv.org/html/2608.04339#bib.bib9)\)using response\-level development losses\. For each candidate system prompt, we construct a Bonferroni\-corrected one\-sided KS upper bound on CVaR at 25% and select the prompt with the smallest bound\. Because the controlled question variants are not fully independent, we use PRC as a comparative selector rather than claiming nominal coverage\. We evaluate the system prompts selected by Average selection, Pure GroupDRO, and PRC, along with the weights selected by Constrained Mixed GroupDRO and the unconstrained Mixed GroupDRO comparator, on held\-out test questions\.

We report three evaluation statistics\. The Overall Mean is averaged over all 72 metric\-group pairs, the Worst 25% Mean is averaged over the 18 highest\-loss pairs, and the worst\-case score is the maximum score among all 72 pairs\. We also compare Constrained Mixed GroupDRO with Average selection across all model\-question\-variant\-metric scores in the full evaluation set\. This analysis reports how many scores improve, worsen, or remain unchanged, the mean change among the improved and worsened scores, and the overall net change\.

For the candidate\-growth analysis, we sample 500 system\-prompt subsets of each sizeK∈\{5,15,25,35,45\}K\\in\\\{5,15,25,35,45\\\}from the same fixed pool of 55 system prompts and refit the selectors on each subset\. ForK=55K=55, we use the full pool once\. The development and test questions remain unchanged\. We then track how the Overall Mean, Worst 25% Mean, and worst\-case score change asKKgrows\.

### 4\.4Validating the Evaluation Groups

A worst\-group objective is meaningful only if the groups reflect real differences in response quality\. The grouping factors are task\-motivated: language, domain\-literacy signal, and question framing describe how the same question is phrased\. They are not result\-driven groups\. Empirically, both consumer\-finance and MIRA baseline responses show clear gaps between average and worst\-group scores\. The gaps are positive across all model\-domain\-metric settings, with detailed results reported in Appendix[D\.1](https://arxiv.org/html/2608.04339#A4.SS1)\. We therefore use the 24 groups to measure robustness across different ways of phrasing the same question\.

Table 1:Controlled score differences pooled across five models\. Reference groups are shown in parentheses\.To validate the evaluation groups, we fit fixed\-effects regressions on development system\-prompt\-pool responses, pooling over five response\-generating models and controlling for seed question, system prompt, and model, with standard errors clustered by seed question\. Table[1](https://arxiv.org/html/2608.04339#S4.T1)reports representative coefficients\. The strongest and most consistent effects come from literacy signals, while question structure, language, and asset/register factors also affect scores in domain\-specific ways\. This supports our use of full evaluation groups for joint worst\-case system\-prompt selection\.

A matched\-pair placebo test further supports the literacy results\. In both domains, the observed low–high literacy differences exceed all 1000 random label flips \(pperm=1/1001<0\.001p\_\{\\mathrm\{perm\}\}=1/1001<0\.001\)\. Detailed results are reported in Appendix[D\.3](https://arxiv.org/html/2608.04339#A4.SS3)\.

Our groups are based on how the question is phrased, not on inferred user identity\. We do not ask the model to infer a person’s real education level, income, health status, or financial expertise\. Instead, we vary the phrasing of the same information need while keeping the underlying checklist fixed\. We use fairness language only for the literacy factor because low health\- or financial\-literacy signals may come from people who are less able to fill in missing information on their own\. Other factors, such as language, register, skeleton, asset level, and question frame, are used to measure robustness to different ways of asking the same question\.

### 4\.5Results

#### Constrained Mixed GroupDRO Improves Average and Worst\-Case Quality

Tables[2](https://arxiv.org/html/2608.04339#S4.T2)and[3](https://arxiv.org/html/2608.04339#S4.T3)show that Constrained Mixed GroupDRO improves both average and worst\-case quality over the no\-mitigation baseline for all five models in both domains\. All results use a single set of system\-prompt weights jointly optimized over the 72 metric–group pairs, with development mean loss constrained to at most 0\.5% above Average selection\.

The effect is especially clear in consumer finance, where Constrained Mixed GroupDRO further improves answer quality even when baseline scores are already low\. For Qwen36Plus, the mean/worst scores decrease from 1\.421/1\.750 to 1\.266/1\.513\. For DeepSeek\-V4Pro, they decrease from 1\.497/1\.950 to 1\.280/1\.458\. The same overall pattern holds for the remaining models and in MIRA\.

Table 2:Held\-out MIRA test results\.Table 3:Held\-out consumer\-finance test results\.
#### Mean\-Tail Pareto Tradeoff

Constrained Mixed GroupDRO introduces a tradeoff between mean quality and protection of poorly performing metric\-group pairs\. We vary the allowed development mean increase over Average selection across\{0%,0\.5%,1%,2%\}\\\{0\\%,0\.5\\%,1\\%,2\\%\\\}and the unconstrained setting\. For each setting, we plot the overall mean score against the mean score of the worst\-performing 25% of metric\-group pairs, averaged over five models\. Lower values on both axes are better\. This shows how much the weakest 25% improve for a given change in average quality\. The optimization itself still minimizes the joint worst\-case score\. We also include PRC and Pure GroupDRO as single\-system\-prompt comparators\. Because neither comparator variesϵ\\epsilon, they are shown as separate points\.

Figure[1](https://arxiv.org/html/2608.04339#S4.F1)shows the held\-out mean\-tail tradeoff for consumer finance and MIRA\. The corresponding development analysis is reported in Appendix[I](https://arxiv.org/html/2608.04339#A9)\. Constrained Mixed \(0\.5%\) provides a strong balance between overall mean quality and the Worst 25% mean on both development and held\-out data\. In consumer finance, it achieves the lowest overall mean and worst\-25% scores among the tested settings\. In MIRA, it substantially improves the worst\-25% mean while keeping the overall mean close to Average selection\. It also achieves lower overall mean and worst\-25% mean than PRC and Pure GroupDRO in both domains\. We therefore use the 0\.5% constraint as the main setting in our experiments\.

To examine whether these tail improvements come at the expense of other scores, Table[4](https://arxiv.org/html/2608.04339#S4.T4)compares Constrained Mixed GroupDRO with Average selection across the full evaluation set\. Although slightly more scores worsen than improve on held\-out data, the improvements are larger\. The resulting overall net change is−0\.007\-0\.007in consumer finance and\+0\.002\+0\.002in MIRA, showing that the overall mean remains nearly unchanged\. Together with the mean\-tail tradeoff, this shows that Constrained Mixed GroupDRO improves poorly performing metric\-group pairs while keeping overall mean quality close to Average selection\.

![Refer to caption](https://arxiv.org/html/2608.04339v1/pareto_test.png)Figure 1:Held\-out mean\-tail tradeoff for consumer finance and MIRATable 4:Full evaluation set test score change for Constrained Mixed GroupDRO relative to Average selection\. 243 MIRA comparisons with N/A ratings are excluded\.
#### Constrained Mixed GroupDRO Reveals System\-Prompt Complementarity

Throughout this subsection, worst\-case denotes the maximum score among all 72 metric–group pairs, whereas Worst 25% mean denotes the average score across the 18 highest\-loss pairs\.

On development data, Constrained Mixed GroupDRO lowers the joint worst\-case score relative to Average selection for all five models in both domains and relative to Pure GroupDRO in 9 of the 10 model\-domain settings\. The full development results are reported in Appendix[E\.1](https://arxiv.org/html/2608.04339#A5.SS1)\. Every constrained mixed solution assigns positive weight to multiple system prompts, with support sizes ranging from 2 to 6 in MIRA and from 3 to 4 in consumer finance\. This provides direct evidence of complementarity across metric\-group pairs\.

In Tables[2](https://arxiv.org/html/2608.04339#S4.T2)and[3](https://arxiv.org/html/2608.04339#S4.T3), bold marks the best score among Average selection, Pure GroupDRO, PRC, and Constrained Mixed GroupDRO for each statistic, while underlining marks Constrained Mixed overall mean or worst 25% mean scores that are lower than unconstrained Mixed GroupDRO\. On test data, Constrained Mixed GroupDRO lowers the worst\-case score in 4/5 consumer\-finance settings compared with each single\-system\-prompt selector\. In MIRA, the corresponding counts are 4/5, 4/5 and 5/5\. PRC remains competitive\. On MIRA DeepSeek, it improves the held\-out Overall Mean and Worst 25% Mean relative to Average selection, although its Worst score is higher\.

On test data, Constrained Mixed GroupDRO achieves a lower Worst 25% mean than Average selection, Pure GroupDRO, and PRC in all 10 model\-domain settings\. Paired seed\-level permutation tests confirm these differences after Holm correction \(10,000 permutations\. AllpHolm≤\.002p\_\{\\mathrm\{Holm\}\}\\leq\.002\) in Appendix[E\.2](https://arxiv.org/html/2608.04339#A5.SS2)\. The comparison with PRC also shows that response\-level tail control does not provide the same protection as directly optimizing performance across metric–group pairs\. These improvements in worst\-case and Worst 25% quality do not come at a high cost to overall mean quality\. Across all 10 model–domain settings, the Constrained Mixed Overall Mean remains within 0\.026 of the better single\-system\-prompt result and is lower in three settings\.

#### System\-Prompt Weight Analysis

Figure[2](https://arxiv.org/html/2608.04339#S4.F2)shows the weights assigned to individual system prompts by Constrained Mixed GroupDRO for each model and domain\.

![Refer to caption](https://arxiv.org/html/2608.04339v1/prompt_weight.png)Figure 2:System\-prompt weights learned by Constrained Mixed GroupDRO\.Constrained Mixed GroupDRO uses only a few system prompts for each model: 2\-6 in MIRA and 3\-4 in consumer finance\. The top 2 prompts together receive 0\.68\-1\.00 of the weight in MIRA and 0\.70\-0\.91 in consumer finance\. MIRA GLM5 uses only two prompts, with one performing better onm1m\_\{1\}andm2m\_\{2\}and the other onm3m\_\{3\}\. MIRA Llama uses six prompts because actionability, completeness, and no\-referral prompts help different metric\-group pairs\. Of these six prompts, the top three receive 91\.9% of the total weight\.

The family\-level analysis in Appendix[H](https://arxiv.org/html/2608.04339#A8)shows that completeness prompts receive the largest weights in MIRA, while actionability prompts receive the largest weights in consumer finance\. For finance Gemma, removing the highest\-weight prompt increases development worst case loss by 0\.070, while removing the lowest\-weight prompt increases it by only 0\.002\.

To illustrate how the selected prompts affect individual answers, we also present case studies using one held\-out example from each domain in Appendix[F](https://arxiv.org/html/2608.04339#A6), where the highest\-weight Constrained Mixed prompt produces the best\-scoring response in both cases\.

#### Candidate\-Growth Analysis

Our main experiments use a pool of 55 system prompts, but the number of available candidates may vary in practice\. We therefore examine performance as the pool size increases fromK=5K=5toK=55K=55, following the candidate\-growth protocol described in Section[4\.3](https://arxiv.org/html/2608.04339#S4.SS3)\. Figure[3](https://arxiv.org/html/2608.04339#S4.F3)reports the test Overall Mean and Worst 25% Mean for consumer finance and MIRA\.

On test data, Constrained Mixed achieves the lowest Worst 25% Mean across all pool sizes in both domains\. As the pool size increases, its Worst 25% Mean generally decreases, while its Overall Mean remains relatively stable\. Scores often stabilize after about 25 or 35 system prompts, suggesting that moderate\-size pools already provide most of the improvement\. The corresponding development and worst\-case results show similar patterns and are reported in Appendix[G](https://arxiv.org/html/2608.04339#A7)\.

![Refer to caption](https://arxiv.org/html/2608.04339v1/candidate_growth_test.png)Figure 3:Test candidate\-growth results for consumer finance and MIRA

## 5Conclusion

Equivalent questions can receive LLM answers of different quality depending on how they are phrased\. We show that this problem can be mitigated at the system\-prompt selection stage\. Constrained Mixed GroupDRO assigns weights to system prompts in an existing pool to improve joint worst\-case quality across evaluation metrics and groups while keeping average quality nearly unchanged\. Our results show that different system prompts help different parts of the evaluation set and that moderate\-size pools provide much of the full\-pool benefit\. Overall, the method offers a practical way to make LLM information\-seeking systems more robust to differences in question phrasing\.

## References

- L\. A\. Agrawal, S\. Tan, D\. Soylu, N\. Ziems, R\. Khare, K\. Opsahl\-Ong, A\. Singhvi, H\. Shandilya, M\. J\. Ryan, M\. Jiang, C\. Potts, K\. Sen, A\. Dimakis, I\. Stoica, D\. Klein, M\. Zaharia, and O\. Khattab \(2026\)GEPA: reflective prompt evolution can outperform reinforcement learning\.InThe Fourteenth International Conference on Learning Representations,Cited by:[Appendix A](https://arxiv.org/html/2608.04339#A1.p1.1),[§1](https://arxiv.org/html/2608.04339#S1.p2.1)\.
- Low health literacy and health outcomes: an updated systematic review\.Annals of Internal Medicine155\(2\),pp\. 97–107\.External Links:[Document](https://dx.doi.org/10.7326/0003-4819-155-2-201107190-00005)Cited by:[§1](https://arxiv.org/html/2608.04339#S1.p1.1),[§2](https://arxiv.org/html/2608.04339#S2.SS0.SSS0.Px3.p1.1)\.
- DeepSeek AI \(2026\)DeepSeek\-V4: towards highly efficient million\-token context intelligence\.Technical reportDeepSeek\.Note:Technical report\. Accessed: 2026\-05\-20External Links:[Link](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)Cited by:[§4\.3](https://arxiv.org/html/2608.04339#S4.SS3.p1.1)\.
- M\. Deng, J\. Wang, C\. Hsieh, Y\. Wang, H\. Guo, T\. Shu, M\. Song, E\. Xing, and Z\. Hu \(2022\)RLPrompt: optimizing discrete text prompts with reinforcement learning\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,Y\. Goldberg, Z\. Kozareva, and Y\. Zhang \(Eds\.\),Abu Dhabi, United Arab Emirates,pp\. 3369–3391\.External Links:[Link](https://aclanthology.org/2022.emnlp-main.222/),[Document](https://dx.doi.org/10.18653/v1/2022.emnlp-main.222)Cited by:[Appendix A](https://arxiv.org/html/2608.04339#A1.p1.1)\.
- E\. Diana, W\. Gill, M\. Kearns, K\. Kenthapadi, and A\. Roth \(2021\)Minimax group fairness: algorithms and experiments\.InProceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society,AIES ’21,New York, NY, USA,pp\. 66–76\.External Links:ISBN 9781450384735,[Link](https://doi.org/10.1145/3461702.3462523),[Document](https://dx.doi.org/10.1145/3461702.3462523)Cited by:[§1](https://arxiv.org/html/2608.04339#S1.p4.1),[§2](https://arxiv.org/html/2608.04339#S2.SS0.SSS0.Px1.p1.1)\.
- V\. Do, X\. Nguyen, V\. Hoang, D\. Nguyen, S\. Sabahi, J\. Yang, H\. Hotta, M\. Nguyen, and H\. Le \(2025\)Automatic prompt selection for large language models\.InPacific\-Asia Conference on Knowledge Discovery and Data Mining,pp\. 91–102\.Cited by:[Appendix A](https://arxiv.org/html/2608.04339#A1.p1.1)\.
- C\. Fernando, D\. Banarse, H\. Michalewski, S\. Osindero, and T\. Rocktäschel \(2023\)Promptbreeder: self\-referential self\-improvement via prompt evolution\.arXiv preprint arXiv:2309\.16797\.Cited by:[Appendix A](https://arxiv.org/html/2608.04339#A1.p1.1)\.
- GLM\-5\-Team \(2026\)GLM\-5: from vibe coding to agentic engineering\.External Links:2602\.15763,[Link](https://arxiv.org/abs/2602.15763)Cited by:[§4\.3](https://arxiv.org/html/2608.04339#S4.SS3.p1.1)\.
- Q\. Guo, R\. Wang, J\. Guo, B\. Li, K\. Song, X\. Tan, G\. Liu, J\. Bian, and Y\. Yang \(2024\)Connecting large language models with evolutionary algorithms yields powerful prompt optimizers\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 34133–34156\.Cited by:[Appendix A](https://arxiv.org/html/2608.04339#A1.p1.1),[§1](https://arxiv.org/html/2608.04339#S1.p2.1)\.
- S\. Gupta, V\. Shrivastava, A\. Deshpande, A\. Kalyan, P\. Clark, A\. Sabharwal, and T\. Khot \(2024\)Bias runs deep: implicit reasoning biases in persona\-assigned LLMs\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=kGteeZ18Ir)Cited by:[§1](https://arxiv.org/html/2608.04339#S1.p1.1),[§2](https://arxiv.org/html/2608.04339#S2.SS0.SSS0.Px3.p1.1)\.
- T\. B\. Hashimoto, M\. Srivastava, H\. Namkoong, and P\. Liang \(2018\)Fairness without demographics in repeated loss minimization\.External Links:1806\.08010,[Link](https://arxiv.org/abs/1806.08010)Cited by:[§1](https://arxiv.org/html/2608.04339#S1.p4.1),[§2](https://arxiv.org/html/2608.04339#S2.SS0.SSS0.Px1.p1.1)\.
- V\. Hofmann, P\. R\. Kalluri, D\. Jurafsky, and S\. King \(2024\)AI generates covertly racist decisions about people based on their dialect\.Nature633\(8028\),pp\. 147–154\.External Links:[Document](https://dx.doi.org/10.1038/s41586-024-07856-5),[Link](https://doi.org/10.1038/s41586-024-07856-5)Cited by:[§1](https://arxiv.org/html/2608.04339#S1.p1.1),[§2](https://arxiv.org/html/2608.04339#S2.SS0.SSS0.Px3.p1.1)\.
- B\. Lester, R\. Al\-Rfou, and N\. Constant \(2021\)The power of scale for parameter\-efficient prompt tuning\.InProceedings of the 2021 conference on empirical methods in natural language processing,pp\. 3045–3059\.Cited by:[Appendix A](https://arxiv.org/html/2608.04339#A1.p1.1)\.
- X\. L\. Li and P\. Liang \(2021\)Prefix\-tuning: optimizing continuous prompts for generation\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),C\. Zong, F\. Xia, W\. Li, and R\. Navigli \(Eds\.\),Online,pp\. 4582–4597\.External Links:[Link](https://aclanthology.org/2021.acl-long.353/),[Document](https://dx.doi.org/10.18653/v1/2021.acl-long.353)Cited by:[Appendix A](https://arxiv.org/html/2608.04339#A1.p1.1)\.
- X\. Liu, Y\. Zheng, Z\. Du, M\. Ding, Y\. Qian, Z\. Yang, and J\. Tang \(2023\)GPT understands, too\.External Links:2103\.10385,[Link](https://arxiv.org/abs/2103.10385)Cited by:[Appendix A](https://arxiv.org/html/2608.04339#A1.p1.1)\.
- Meta AI \(2025\)The Llama 4 herd: the beginning of a new era of natively multimodal AI innovation\.Note:Accessed: 2026\-07\-15External Links:[Link](https://ai.meta.com/blog/llama-4-multimodal-intelligence/)Cited by:[§4\.3](https://arxiv.org/html/2608.04339#S4.SS3.p1.1)\.
- OpenAI \(2026a\)Introducing GPT\-5\.4 mini and nano\.Note:[https://openai\.com/index/introducing\-gpt\-5\-4\-mini\-and\-nano/](https://openai.com/index/introducing-gpt-5-4-mini-and-nano/)Cited by:[§4\.3](https://arxiv.org/html/2608.04339#S4.SS3.p1.1)\.
- OpenAI \(2026b\)Introducing GPT\-5\.4\.Note:[https://openai\.com/index/introducing\-gpt\-5\-4/](https://openai.com/index/introducing-gpt-5-4/)Cited by:[§4\.1](https://arxiv.org/html/2608.04339#S4.SS1.p2.1)\.
- K\. Opsahl\-Ong, M\. J\. Ryan, J\. Purtell, D\. Broman, C\. Potts, M\. Zaharia, and O\. Khattab \(2024\)Optimizing instructions and demonstrations for multi\-stage language model programs\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 9340–9366\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.525/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.525)Cited by:[Appendix A](https://arxiv.org/html/2608.04339#A1.p1.1)\.
- A\. Prasad, P\. Hase, X\. Zhou, and M\. Bansal \(2023\)GrIPS: gradient\-free, edit\-based instruction search for prompting large language models\.External Links:2203\.07281,[Link](https://arxiv.org/abs/2203.07281)Cited by:[Appendix A](https://arxiv.org/html/2608.04339#A1.p1.1)\.
- S\. Sagawa, P\. W\. Koh, T\. B\. Hashimoto, and P\. Liang \(2020\)Distributionally robust neural networks\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=ryxGuJrFvS)Cited by:[§1](https://arxiv.org/html/2608.04339#S1.p4.1),[§2](https://arxiv.org/html/2608.04339#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2608.04339#S3.SS1.SSS0.Px2.p2.1)\.
- P\. G\. Sessa, I\. Bogunovic, M\. Kamgarpour, and A\. Krause \(2020\)Mixed strategies for robust optimization of unknown objectives\.InInternational Conference on Artificial Intelligence and Statistics,pp\. 2970–2980\.Cited by:[§3\.2](https://arxiv.org/html/2608.04339#S3.SS2.p1.1)\.
- T\. Shin, Y\. Razeghi, R\. L\. L\. Iv, E\. Wallace, and S\. Singh \(2020\)Autoprompt: eliciting knowledge from language models with automatically generated prompts\.InProceedings of the 2020 conference on empirical methods in natural language processing \(EMNLP\),pp\. 4222–4235\.Cited by:[Appendix A](https://arxiv.org/html/2608.04339#A1.p1.1)\.
- T\. Sun, Y\. Shao, H\. Qian, X\. Huang, and X\. Qiu \(2022\)Black\-box tuning for language\-model\-as\-a\-service\.InInternational Conference on Machine Learning,pp\. 20841–20855\.Cited by:[Appendix A](https://arxiv.org/html/2608.04339#A1.p1.1)\.
- G\. Team, S\. E\. Abd, V\. Aggarwal, R\. Algayres, A\. Andreev, O\. Bachem, I\. Ballantyne, C\. Brick, V\. Cărbune, M\. Casbon, M\. Chaturvedi, V\. Cotruta, A\. Coucke, P\. Culliton, R\. Dadashi, L\. Dixon, M\. Elhawaty, U\. Evci, C\. Farabet, J\. Ferret, F\. Galgani, S\. Girgin, J\. Grill, M\. Grootendorst, J\. Guo, C\. Hardin, Y\. He, S\. M\. Hernandez, O\. Homburger, L\. Hussenot, J\. Ji, A\. Joulin, A\. Kamath, P\. Kassraie, O\. Lacombe, P\. Lahoti, G\. Liu, G\. Martins, L\. Martins, T\. Matejovicova, R\. Merhej, N\. Momchev, S\. Mondal, R\. Mullins, S\. R\. Panyam, S\. Pathak, S\. Perrin, A\. S\. Pinto, E\. Pot, A\. Pouget, A\. Ramé, S\. Ramos, D\. Reid, D\. Rim, M\. Rivière, K\. Roth, L\. Rouillard, O\. Sanseviero, P\. G\. Sessa, S\. Settle, D\. Sinopalnikov, S\. Smoot, P\. Stanczyk, A\. Steiner, L\. Stewart, I\. Tolstikhin, M\. Tschannen, A\. Tsitsulin, N\. Vieillard, R\. Wu, P\. Xu, H\. Yang, E\. Yvinec, L\. Zhang, J\. Zou, N\. Aagnes, A\. Abdelhamed, S\. Agrawal, S\. Agrawal, I\. Alabdulmohsin, J\. B\. Alayrac, U\. Alon, C\. Amarnath, A\. Anand, C\. Anastasiou, S\. Ariafar, F\. Aubet, K\. Axiotis, F\. Barbero, J\. Barral, A\. Bendebury, U\. Bergmann, S\. Bileschi, K\. Black, M\. Blondel, S\. Borgeaud, A\. Bražinskas, R\. Burnell, R\. Busa\-Fekete, M\. Cai, G\. Cameron, C\. Caucheteux, G\. Chadha, J\. Chan, A\. Chawla, B\. J\. Chen, J\. Chen, L\. Chen, X\. Chen, D\. Cheng, T\. Chien, N\. Chinaev, Y\. Chou, Z\. Chu, B\. Coleman, P\. Consul, S\. Conway\-Rahman, S\. Crowell, D\. Cutler, V\. Dani, S\. Daruki, A\. Das, D\. Deutsch, N\. Dikkala, L\. Ding, Q\. Ding, S\. Dodhia, K\. Donhauser, T\. Doshi, A\. Dragan, A\. Druinsky, S\. Dua, Z\. Egyed, D\. Eisenbud, D\. Eppens, C\. Fan, B\. Fatemi, Y\. Fathullah, V\. Feinberg, M\. Ferev, T\. Fujimoto, I\. Galatzer\-Levy, J\. Gante, S\. Geisler, S\. Ghosal, A\. M\. Girgis, A\. Go, A\. Gokhale, A\. Grills, Y\. Gu, P\. Gupta, G\. Guruganesh, R\. Hadsell, H\. Harkous, J\. Harlalka, D\. Hassabis, A\. Hauth, J\. Heyward, A\. Hosseini, C\. Hsia, I\. Hsu, X\. Huang, Y\. Huang, K\. Hui, A\. Hutter, T\. I, F\. Iliopoulos, A\. Jain, G\. Jawahar, Z\. Ji, Q\. Jin, M\. Johnson, K\. Joshi, A\. Kandoor, W\. Kang, K\. Kavukcuoglu, M\. Kazemi, K\. Kenealy, A\. Khalifa, P\. Kirk, S\. Kothawade, V\. Kovalev, N\. Kovelamudi, A\. Kraft, R\. Kumar, H\. Kuppam, J\. Lannin, C\. Lee, S\. Lee, D\. Lepikhin, D\. Li, Q\. Li, V\. Liévin, E\. Lin, Z\. Lin, C\. Liu, T\. Liu, T\. Liu, X\. Liu, M\. Lunayach, M\. Ma, G\. Madan, A\. Maksai, E\. Malmi, M\. Matuszak, D\. McDuff, G\. Menghani, D\. Mirylenka, K\. Misiunas, V\. Misra, A\. Mitran, K\. Mohamed, M\. Mukha, E\. Noland, J\. O’Donnell, K\. Olszewska, B\. Orlando, W\. Pan, R\. Panigrahy, U\. Parekh, C\. Park, E\. Paskie, L\. Peng, B\. Petrini, S\. Petrov, J\. Pfeiffer, B\. Piot, M\. Plomecka, S\. Poder, O\. Ponce, A\. Pramanik, D\. Racz, A\. Rajan, M\. Ramanovich, A\. Rao, M\. Ritter, V\. Rodrigues, E\. Rosen, M\. Rybiński, N\. Sachdeva, M\. E\. Sander, R\. Sathyanarayana, S\. Savla, S\. Schmidgall, T\. Schuster, B\. Seguin, A\. Sellergren, A\. Severyn, I\. Shafran, D\. Shah, Y\. Shangguan, A\. Shenoy, P\. Shenoy, R\. Shivanna, P\. Sho, L\. Spangher, W\. Stokowiec, T\. Strother, Y\. Su, Y\. Sun, M\. Sundararajan, A\. Tacchetti, M\. H\. Taege, P\. Tafti, C\. Tekur, R\. Thapa, M\. Traverse, L\. Treven, T\. Tu, C\. T\. Tung, P\. Veličković, M\. P\. Venkat, S\. G\. Venkatesh, V\. Venkiteswaran, F\. Visin, A\. Vitvitskyi, K\. Vodrahalli, W\. Wang, X\. Wang, T\. Warkentin, J\. Wassenberg, J\. Wieting, L\. Xiao, H\. Xu, Y\. Xu, F\. Xue, A\. Yadav, J\. Yan, A\. Yang, L\. Yang, M\. Yang, Z\. Ying, J\. H\. Yoo, S\. Zafar, F\. Zhang, J\. Zhang, J\. Zhang, X\. Zhang, C\. Zhao, D\. Zhou, and C\. Zou \(2026\)Gemma 4 technical report\.External Links:2607\.02770,[Link](https://arxiv.org/abs/2607.02770)Cited by:[§4\.3](https://arxiv.org/html/2608.04339#S4.SS3.p1.1)\.
- J\. v\. Neumann \(1928\)Zur theorie der gesellschaftsspiele\.Mathematische Annalen100\(1\),pp\. 295–320\.External Links:[Document](https://dx.doi.org/10.1007/BF01448847),[Link](https://doi.org/10.1007/BF01448847)Cited by:[§3\.2](https://arxiv.org/html/2608.04339#S3.SS2.p1.1)\.
- Y\. Wen, N\. Jain, J\. Kirchenbauer, M\. Goldblum, J\. Geiping, and T\. Goldstein \(2023\)Hard prompts made easy: gradient\-based discrete optimization for prompt tuning and discovery\.Advances in Neural Information Processing Systems36,pp\. 51008–51025\.Cited by:[Appendix A](https://arxiv.org/html/2608.04339#A1.p1.1)\.
- M\. Xu, Q\. Yang, Q\. Wang, X\. Dai, W\. Wu, and C\. Gao \(2026\)MIRA: a bilingual benchmark for medical information response audit\.External Links:2605\.28025,[Link](https://arxiv.org/abs/2605.28025)Cited by:[§1](https://arxiv.org/html/2608.04339#S1.p1.1),[§1](https://arxiv.org/html/2608.04339#S1.p5.1),[§2](https://arxiv.org/html/2608.04339#S2.SS0.SSS0.Px3.p1.1),[§3\.1](https://arxiv.org/html/2608.04339#S3.SS1.SSS0.Px1.p1.9),[§4\.2](https://arxiv.org/html/2608.04339#S4.SS2.p1.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. Qiu \(2025\)Qwen3 technical report\.External Links:2505\.09388,[Document](https://dx.doi.org/10.48550/arXiv.2505.09388),[Link](https://arxiv.org/abs/2505.09388)Cited by:[§4\.3](https://arxiv.org/html/2608.04339#S4.SS3.p1.1)\.
- C\. Yang, X\. Wang, Y\. Lu, H\. Liu, Q\. V\. Le, D\. Zhou, and X\. Chen \(2024\)Large language models as optimizers\.InInternational Conference on Learning Representations,Cited by:[Appendix A](https://arxiv.org/html/2608.04339#A1.p1.1),[§1](https://arxiv.org/html/2608.04339#S1.p2.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2023\)Judging LLM\-as\-a\-Judge with MT\-Bench and Chatbot Arena\.arXiv\.Note:arXiv:2306\.05685 \[cs\.CL\]External Links:[Link](http://arxiv.org/abs/2306.05685),[Document](https://dx.doi.org/10.48550/arXiv.2306.05685)Cited by:[§4\.3](https://arxiv.org/html/2608.04339#S4.SS3.p1.1)\.
- Y\. Zhou, A\. I\. Muresanu, Z\. Han, K\. Paster, S\. Pitis, H\. Chan, and J\. Ba \(2023\)Large language models are human\-level prompt engineers\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=92gvk82DE-)Cited by:[Appendix A](https://arxiv.org/html/2608.04339#A1.p1.1),[§1](https://arxiv.org/html/2608.04339#S1.p2.1)\.
- T\. P\. Zollo, T\. Morrill, Z\. Deng, J\. Snell, T\. Pitassi, and R\. Zemel \(2024\)Prompt risk control: a rigorous framework for responsible deployment of large language models\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=5tGGWOijvq)Cited by:[§2](https://arxiv.org/html/2608.04339#S2.SS0.SSS0.Px2.p1.1),[§4\.3](https://arxiv.org/html/2608.04339#S4.SS3.p3.1)\.

## Appendix AExtended Related Work

Prompt optimization shows that prompt design can substantially affect LLM output quality\. White\-box methods use parameter or gradient access to optimize soft prompts\(Li and Liang[2021](https://arxiv.org/html/2608.04339#bib.bib16); Lesteret al\.[2021](https://arxiv.org/html/2608.04339#bib.bib17); Liuet al\.[2023](https://arxiv.org/html/2608.04339#bib.bib18)\)or search discrete tokens\(Shinet al\.[2020](https://arxiv.org/html/2608.04339#bib.bib19); Wenet al\.[2023](https://arxiv.org/html/2608.04339#bib.bib20)\)\. Black\-box methods search with model feedback only, via derivative\-free optimization\(Sunet al\.[2022](https://arxiv.org/html/2608.04339#bib.bib21)\), reinforcement learning\(Denget al\.[2022](https://arxiv.org/html/2608.04339#bib.bib22)\), or edit\-based search\(Prasadet al\.[2023](https://arxiv.org/html/2608.04339#bib.bib23)\), and recent LLM\-as\-optimizer and evolutionary methods generate prompt candidates automatically\(Zhouet al\.[2023](https://arxiv.org/html/2608.04339#bib.bib7); Yanget al\.[2024](https://arxiv.org/html/2608.04339#bib.bib13); Fernandoet al\.[2023](https://arxiv.org/html/2608.04339#bib.bib14); Guoet al\.[2024](https://arxiv.org/html/2608.04339#bib.bib24); Agrawalet al\.[2026](https://arxiv.org/html/2608.04339#bib.bib15); Opsahl\-Onget al\.[2024](https://arxiv.org/html/2608.04339#bib.bib25)\)\. A complementary line selects prompts from a candidate set rather than generating new text, ranking candidates by validation performance or learned evaluators\(Doet al\.[2025](https://arxiv.org/html/2608.04339#bib.bib34)\)\.

## Appendix BProof of Proposition[1](https://arxiv.org/html/2608.04339#Thmproposition1)

###### Proof\.

Each feasible single system promptpi∈𝒫ϵp\_\{i\}\\in\\mathcal\{P\}\_\{\\epsilon\}corresponds to the one\-hot vectorei∈ΔNe\_\{i\}\\in\\Delta\_\{N\}, which places all weight onpip\_\{i\}\. By the definition of𝒫ϵ\\mathcal\{P\}\_\{\\epsilon\},

L¯​\(ei\)=1\|ℳ\|​\|𝒢\|​∑m∈ℳ∑g∈𝒢Rm,g​\(ei\)≤\(1\+ϵ\)​L¯avg\.\\bar\{L\}\(e\_\{i\}\)=\\frac\{1\}\{\|\\mathcal\{M\}\|\|\\mathcal\{G\}\|\}\\sum\_\{m\\in\\mathcal\{M\}\}\\sum\_\{g\\in\\mathcal\{G\}\}R\_\{m,g\}\(e\_\{i\}\)\\leq\(1\+\\epsilon\)\\bar\{L\}\_\{\\mathrm\{avg\}\}\.Therefore,ei∈ΔN,ϵe\_\{i\}\\in\\Delta\_\{N,\\epsilon\}\. Thus, the one\-hot vectors corresponding to feasible single\-system\-prompt choices form a subset of the constrained mixed feasible set, and minimizing over the larger set givesVmix,ϵ≤Vpure,ϵV\_\{\\mathrm\{mix\},\\epsilon\}\\leq V\_\{\\mathrm\{pure\},\\epsilon\}\.

Both feasible sets are nonempty\. Because the losses are nonnegative andϵ≥0\\epsilon\\geq 0,

L¯avg≤\(1\+ϵ\)​L¯avg\.\\bar\{L\}\_\{\\mathrm\{avg\}\}\\leq\(1\+\\epsilon\)\\bar\{L\}\_\{\\mathrm\{avg\}\}\.So,pavg∈𝒫ϵp\_\{\\mathrm\{avg\}\}\\in\\mathcal\{P\}\_\{\\epsilon\}and its corresponding one\-hot vector belongs toΔN,ϵ\\Delta\_\{N,\\epsilon\}\.

Because𝒫ϵ\\mathcal\{P\}\_\{\\epsilon\}is finite and nonempty, a constrained single\-prompt minimizer exists\. The setΔN,ϵ\\Delta\_\{N,\\epsilon\}is a closed subset of the compact simplexΔN\\Delta\_\{N\}and is therefore compact\. Since

w↦maxm∈ℳg∈𝒢⁡Rm,g​\(w\)w\\mapsto\\max\_\{\\begin\{subarray\}\{c\}m\\in\\mathcal\{M\}\\\\ g\\in\\mathcal\{G\}\\end\{subarray\}\}R\_\{m,g\}\(w\)is continuous, an optimal constrained mixed solution also exists\.

First supposeVmix,ϵ=Vpure,ϵV\_\{\\mathrm\{mix\},\\epsilon\}=V\_\{\\mathrm\{pure\},\\epsilon\}\. Let

pi⋆∈arg​minpi∈𝒫ϵ⁡maxm∈ℳg∈𝒢⁡Rm,g​\(ei\)p\_\{i^\{\\star\}\}\\in\\operatorname\*\{arg\\,min\}\_\{p\_\{i\}\\in\\mathcal\{P\}\_\{\\epsilon\}\}\\max\_\{\\begin\{subarray\}\{c\}m\\in\\mathcal\{M\}\\\\ g\\in\\mathcal\{G\}\\end\{subarray\}\}R\_\{m,g\}\(e\_\{i\}\)be a constrained single\-system\-prompt minimizer\. Thenei⋆∈ΔN,ϵe\_\{i^\{\\star\}\}\\in\\Delta\_\{N,\\epsilon\}, and its objective value is

maxm∈ℳg∈𝒢⁡Rm,g​\(ei⋆\)=Vpure,ϵ=Vmix,ϵ\.\\max\_\{\\begin\{subarray\}\{c\}m\\in\\mathcal\{M\}\\\\ g\\in\\mathcal\{G\}\\end\{subarray\}\}R\_\{m,g\}\(e\_\{i^\{\\star\}\}\)=V\_\{\\mathrm\{pure\},\\epsilon\}=V\_\{\\mathrm\{mix\},\\epsilon\}\.Thus,ei⋆e\_\{i^\{\\star\}\}is an optimal constrained mixed solution that places all weight on a single prompt in𝒫ϵ\\mathcal\{P\}\_\{\\epsilon\}\.

Conversely, suppose an optimal constrained mixed solution is one\-hot, sayeie\_\{i\}\. Its feasibility impliespi∈𝒫ϵp\_\{i\}\\in\\mathcal\{P\}\_\{\\epsilon\}, sinceL¯​\(ei\)\\bar\{L\}\(e\_\{i\}\)equals the mean loss of promptpip\_\{i\}\. Therefore,

Vpure,ϵ≤maxm∈ℳg∈𝒢⁡Rm,g​\(ei\)=Vmix,ϵ\.V\_\{\\mathrm\{pure\},\\epsilon\}\\leq\\max\_\{\\begin\{subarray\}\{c\}m\\in\\mathcal\{M\}\\\\ g\\in\\mathcal\{G\}\\end\{subarray\}\}R\_\{m,g\}\(e\_\{i\}\)=V\_\{\\mathrm\{mix\},\\epsilon\}\.Together withVmix,ϵ≤Vpure,ϵV\_\{\\mathrm\{mix\},\\epsilon\}\\leq V\_\{\\mathrm\{pure\},\\epsilon\}, this givesVmix,ϵ=Vpure,ϵV\_\{\\mathrm\{mix\},\\epsilon\}=V\_\{\\mathrm\{pure\},\\epsilon\}\.

Finally, supposeVmix,ϵ<Vpure,ϵV\_\{\\mathrm\{mix\},\\epsilon\}<V\_\{\\mathrm\{pure\},\\epsilon\}\. Every feasible one\-hot vectoreie\_\{i\}corresponds to somepi∈𝒫ϵp\_\{i\}\\in\\mathcal\{P\}\_\{\\epsilon\}and therefore has objective value

maxm∈ℳg∈𝒢⁡Rm,g​\(ei\)≥Vpure,ϵ\>Vmix,ϵ\.\\max\_\{\\begin\{subarray\}\{c\}m\\in\\mathcal\{M\}\\\\ g\\in\\mathcal\{G\}\\end\{subarray\}\}R\_\{m,g\}\(e\_\{i\}\)\\geq V\_\{\\mathrm\{pure\},\\epsilon\}\>V\_\{\\mathrm\{mix\},\\epsilon\}\.
Hence, no feasible one\-hot vector can be optimal\. Since an optimal constrained mixed solution exists and cannot be one\-hot, every optimal constrained mixed solution places positive weight on at least two system prompts\. ∎

## Appendix CConsumer\-Finance Scoring Agreement

Table[5](https://arxiv.org/html/2608.04339#A3.T5)reports agreement between the trained finance annotator and the LLM judge\. Exact agreement is the proportion of identical ratings, adjacent agreement is the proportion that differ by at most one point, and QWK denotes quadratic weighted kappa\.

Table 5:Consumer\-finance scoring agreement on the 250\-response audit set\.
## Appendix DFull Group Validity Results

This section provides the complete results supporting the group\-validity analysis in Section[4\.4](https://arxiv.org/html/2608.04339#S4.SS4)\. We first report the baseline worst\-minus\-mean gaps and then present the full fixed\-effect regression results for each domain\.

### D\.1Baseline Worst\-Minus\-Mean Gaps

Table[6](https://arxiv.org/html/2608.04339#A4.T6)reports the baseline worst\-minus\-mean gap for each model, domain, and evaluation metric\. Each gap is calculated by subtracting the mean loss across the 24 evaluation groups from the highest group loss\. Larger gaps indicate greater variation in response quality across groups\.

For example, in consumer finance, Llama4Scout has gaps of 0\.607 onm1m\_\{1\}and 0\.624 onm2m\_\{2\}\. In MIRA, DeepSeek\-V4Pro has a gap of 0\.280 onm2m\_\{2\}, and Llama4Scout has a gap of 0\.284 onm3m\_\{3\}\. The gaps are positive across all model\-domain\-metric settings, showing that the worst\-performing group consistently has higher loss than the group mean\. This provides additional motivation for optimizing worst\-group performance rather than relying only on average quality\.

Table 6:Baseline worst\-minus\-mean gaps by model, domain, and evaluation metric\.
### D\.2Controlled Group Differences

Tables[7](https://arxiv.org/html/2608.04339#A4.T7)and[8](https://arxiv.org/html/2608.04339#A4.T8)report the full regression results for MIRA and consumer finance\. ThePooledrows pool all five models, while the remaining rows report model\-specific results\. The regressions control for seed question and system prompt, and the pooled regressions also control for the response\-generating model\. Standard errors are clustered by seed question\. Positive coefficients indicate worse outcomes relative to the reference group\.

Together, these results show that the evaluation groups capture meaningful variation in response quality\. Literacy signals produce the strongest and most consistent differences, while language, question structure, resource constraints, and question framing show domain\-specific effects\. This supports the use of these groups for joint worst\-case system\-prompt selection\.

Table 7:Full controlled score differences for MIRA evaluation groups\. Reference groups are shown in parentheses\. Regressions include seed\-question and system\-prompt fixed effects; the pooled regressions also include model fixed effects\.Table 8:Full controlled score differences for consumer\-finance evaluation groups\. Reference groups are shown in parentheses\. Regressions include seed\-question and system\-prompt fixed effects\. The pooled regressions also include model fixed effects\.
### D\.3Literacy Placebo Tests

As a matched\-pair placebo check, we randomly flip the low/high literacy label within matched pairs 1000 times\. In MIRA, the observed low\-high HLS deltas are 0\.118/0\.114/0\.026 onm1/m2/m3m\_\{1\}/m\_\{2\}/m\_\{3\}, compared with placebo 95th percentiles of 0\.005/0\.005/0\.005\. In consumer finance, the observed low\-high FLS deltas are 0\.094/0\.094/0\.060, compared with placebo 95th percentiles of 0\.007/0\.006/0\.004\. In both domains, the observed deltas exceed all 1000 permutations \(pperm=1/1001<0\.001p\_\{\\mathrm\{perm\}\}=1/1001<0\.001\), showing that the literacy effects are not reproduced by random relabeling\.

## Appendix EFull Development\-Set Results

### E\.1Development Results of Selectors

Tables[9](https://arxiv.org/html/2608.04339#A5.T9)and[10](https://arxiv.org/html/2608.04339#A5.T10)report the complete development\-set results for MIRA and consumer finance\. Each table reports the Overall Mean \(All\), Worst 25% Mean \(25%\), and worst\-case score \(Worst\)\. Lower values are better\.

Constrained Mixed GroupDRO achieves a lower Worst 25% Mean than Average selection, PRC, and Pure GroupDRO in all 10 model–domain settings\. It also achieves a lower worst\-case score than all three single\-system\-prompt selectors in 9 of the 10 settings\. The exception is MIRA GLM5, where its score is 2\.422 compared with 2\.400 for Pure GroupDRO\. Its Overall Mean remains within 0\.013 of the best single\-system\-prompt result in every setting\. Bold indicates the best result among the single\-system\-prompt selectors and Constrained Mixed GroupDRO, while underlining indicates that Constrained Mixed GroupDRO outperforms unconstrained Mixed GroupDRO\.

Table 9:MIRA development\-set results\.Table 10:Consumer\-finance development\-set results\.
### E\.2Paired Selector Significance Tests

We use paired seed\-level permutation tests to assess the held\-out differences between Constrained Mixed GroupDRO and each single\-prompt selector\. The held\-out seed question is the permutation unit\. In each of 10,000 permutations, selector labels are swapped within each seed while keeping all 24 question variants and three metric scores together\. We then recompute the 72 metric\-group means and the three evaluation statistics\. We use two\-sided permutationpp\-values and apply Holm correction across the nine reported tests\. Table[11](https://arxiv.org/html/2608.04339#A5.T11)reports the complete results\.

Table 11:Paired seed\-level permutation tests comparing Constrained Mixed GroupDRO with the single\-prompt selectors on held\-out data\. PositiveΔ\\Deltameans lower loss for Constrained Mixed\. Wins count model–domain settings with positiveΔ\\Delta\. Holm correction is applied across the nine reported tests\.
### E\.3Distribution of Development Score Changes

Table[12](https://arxiv.org/html/2608.04339#A5.T12)compares Constrained Mixed GroupDRO with Average selection across all development\-set model\-question\-variant\-metric scores\. The net changes are\+0\.007\+0\.007in consumer finance and\+0\.011\+0\.011in MIRA, showing that the overall development mean changes little in both domains\.

Table 12:Distribution of development score changes across the full evaluation set for Constrained Mixed GroupDRO relative to Average selection\. F and M denote consumer finance and MIRA\. One MIRA comparison with an N/A rating is excluded\.

## Appendix FCase studies

##### Consumer Finance

We consider a held\-out low\-financial\-literacy question: “What is the difference between a family trust and a regular investment trust, and how do they protect, transfer, and grow money?” Average selection chooses a system prompt that emphasizes actionability and asks the model to explain which documents, terms, and numbers the user should check\. The response recommends reviewing a trust deed, prospectus, and fees, but does not clearly explain that suitability depends on the family’s legal and tax circumstances\. It receives scores of4/4/24/4/2onm1/m2/m3m\_\{1\}/m\_\{2\}/m\_\{3\}\.

Pure GroupDRO chooses a system prompt that emphasizes the preservation of the information needed for judgment and stating important limits and risks\. Its response explains the basic purposes of the two trust structures, but omits important legal and return limits and does not provide a clear suitability test\. It receives scores of4/4/34/4/3\.

Constrained Mixed GroupDRO places its largest weight \(w=0\.506w=0\.506\) on a completeness prompt and distributes the remaining weight across two actionability prompts and one anti\-disclaimer/referral\-only prompt\. The response generated by the highest\-weight prompt explains the setup costs, legal and tax conditions, and the information that should be verified\. It receives scores of1/1/21/1/2\.

##### MIRA

We also consider a held\-out low\-health\-literacy question: “What should be considered when choosing foods that raise blood sugar slowly for people with type 2 diabetes?” Average selection chooses a system prompt that emphasizes medical context and completeness\. Its response explains that the glycemic index ranges from zero to one hundred, but does not give the cutoff used to identify low\-GI foods\. It receives scores of2/2/12/2/1\.

Pure GroupDRO chooses a system prompt that explicitly asks the model not to omit criteria, thresholds, or important distinctions\. Its response explains that foods with a lower glycemic index cause a more gradual increase in blood glucose, but again omits the low\-GI cutoff\. It receives scores of2/2/22/2/2\.

The highest\-weight prompt in the Constrained Mixed GroupDRO support \(w=0\.504w=0\.504\) emphasizes definitions, evaluation criteria, and clinical thresholds\. Its response explicitly states that foods with a glycemic index below 55 are classified as low and explains how the threshold, food examples, and portion size relate to post\-meal glucose\. It receives scores of1/1/11/1/1\.

## Appendix GAdditional Candidate\-Growth Results

Figure[4](https://arxiv.org/html/2608.04339#A7.F4)reports the development candidate\-growth results, while Figure[5](https://arxiv.org/html/2608.04339#A7.F5)reports the worst\-case results on development and held\-out test data\. Across both domains, performance generally improves as the candidate pool expands, with most gains appearing byK=25K=25orK=35K=35\.

![Refer to caption](https://arxiv.org/html/2608.04339v1/candidate_growth_dev.png)Figure 4:Development candidate\-growth results for consumer finance and MIRA, averaged over five models\. Lower is better\.![Refer to caption](https://arxiv.org/html/2608.04339v1/candidate_growth_worst.png)Figure 5:Worst\-case candidate\-growth results on development and held\-out test data for consumer finance and MIRA, averaged over five models\. Lower is better\.Table[13](https://arxiv.org/html/2608.04339#A7.T13)further quantifies how much of the improvement fromK=5K=5toK=55K=55is recovered by each smaller pool\. LetWKd,sW\_\{K\}^\{d,s\}denote the Constrained Mixed GroupDRO \(0\.5%\) joint worst\-case score for pool sizeKK, averaged over five models for domainddand splitss\. We compute the recovered improvement as

Recovery⁡\(K\)=100×W5d,s−WKd,sW5d,s−W55d,s\.\\operatorname\{Recovery\}\(K\)=100\\times\\frac\{W\_\{5\}^\{d,s\}\-W\_\{K\}^\{d,s\}\}\{W\_\{5\}^\{d,s\}\-W\_\{55\}^\{d,s\}\}\.Because lower scores are better, the numerator measures the worst\-case reduction obtained at pool sizeKK, while the denominator measures the total reduction obtained by increasing the pool fromK=5K=5toK=55K=55\. By construction, recovery is 0% atK=5K=5and 100% atK=55K=55\. For example, the 56\.24% value for consumer\-finance development data means thatK=25K=25achieves 56\.24% of the total worst\-case improvement obtained with the full pool\. Across both domains,K=25K=25recovers at least 56\.24% of the full\-pool improvement, andK=35K=35recovers at least 73\.98%\. These results show that moderate\-size pools already provide a substantial share of the benefit observed with 55 system prompts\.

Table 13:Percentage of theK=5K=5toK=55K=55improvement in the Constrained Mixed \(0\.5%\) joint worst\-case score recovered by smaller system\-prompt pools, averaged over five models\. By construction,K=5K=5is 0% andK=55K=55is 100%\. Higher is better\.
## Appendix HFamily\-Level Weight Analysis

Figure[6](https://arxiv.org/html/2608.04339#A8.F6)aggregates the Constrained Mixed GroupDRO weights by mitigation family\. For each model and domain, the weight of a family is the sum of the weights assigned to all system prompts in that family\. This shows whether the mixture combines different mitigation goals or different wording variants within the same family\.

In MIRA, the largest weights generally fall on completeness\-related families, although some models also use actionability and prompts that avoid referral\-only answers\. In consumer finance, the weights are more concentrated on actionability\. For GLM5 and Llama in consumer finance, all family\-level weight falls on the actionability family, even though the weight may be distributed across multiple prompts within that family\. Thus, complementarity can arise both across mitigation families and across wording variants within the same family\.

![Refer to caption](https://arxiv.org/html/2608.04339v1/family_weights.png)Figure 6:System\-prompt weights aggregated by family\.
## Appendix IDevelopment Mean\-Tail Pareto Tradeoff

Figure[7](https://arxiv.org/html/2608.04339#A9.F7)reports the development mean–tail tradeoff for both domains\. Lower values on both axes are better\. Constrained Mixed \(0\.5%\) provides a strong balance between the Overall Mean and Worst 25% Mean\. In both domains, no tested setting achieves lower development scores on both measures, supporting our use of the 0\.5% constraint in the main experiments\.

![Refer to caption](https://arxiv.org/html/2608.04339v1/pareto_dev.png)Figure 7:Development mean\-tail tradeoff for consumer finance and MIRA, averaged over five models\. Lower is better\.