MedConstBench: Benchmarking How Constitutions Steer LLMs in Medical Ethics

# MedConstBench: Benchmarking How Constitutions Steer LLMs in Medical Ethics
Source: [https://arxiv.org/html/2610.36513](https://arxiv.org/html/2610.36513)
Zerui HuangHaomiao YangAffiliation:Xinjin District People’s Hospital, ChinaZixia WangAffiliation:Independent Researcher, ChinaAffiliation:University of Exeter, United Kingdomyang\.374960202@gmail\.comhuangruirui92@gmail\.comcdhmyang@126\.comzw483@exeter\.ac\.uk†These authors contributed equally to this work and share first authorship\.\*Corresponding author\.

###### Abstract

As large language models advance rapidly, they are increasingly used to answer medical questions, and many such models are now deployed locally\. Once a model is deployed, its behavior is governed mainly by the ethical policy written into its constitution\. However, most existing work focuses on factual correctness rather than the trade\-off between competing ethical values\. Even benchmarks that do evaluate ethics treat it as a fixed property of the model rather than the constitution as a variable, so how a constitution reshapes medical\-ethics behavior remains largely unexplored\. We introduce MedConstBench, a medical\-ethics benchmark that treats the deployment constitution as a controlled variable, is organized around Beauchamp’s four principles, and is scored by human\-calibrated multi\-model peer review\. It includes MedDilemma, our own dataset of clinical dilemmas, and supports a systematic analysis of how constitutions steer LLMs\. Evaluating ten open\-source LLMs under four constitutions, we find that the effect of a constitution is dimension\-selective and model\-dependent, and that no constitution is universally optimal\. Motivated by these findings, we propose a do\-no\-harm adaptive selector that chooses a constitution per model, matching the per\-model oracle while guaranteeing a worst\-case floor and generalizing to unseen models at low cost\.

###### Index Terms:

benchmark, large language models, medical ethics, constitutional AI, LLM\-as\-a\-judge

## IIntroduction

Large language models \(LLMs\) have advanced rapidly and are moving from general\-purpose assistants into specialized domains\[[1](https://arxiv.org/html/2610.36513#bib.bib20),[2](https://arxiv.org/html/2610.36513#bib.bib41),[3](https://arxiv.org/html/2610.36513#bib.bib42),[4](https://arxiv.org/html/2610.36513#bib.bib43),[5](https://arxiv.org/html/2610.36513#bib.bib40)\]\. In healthcare\[[6](https://arxiv.org/html/2610.36513#bib.bib17),[7](https://arxiv.org/html/2610.36513#bib.bib47),[8](https://arxiv.org/html/2610.36513#bib.bib44),[9](https://arxiv.org/html/2610.36513#bib.bib45),[10](https://arxiv.org/html/2610.36513#bib.bib46)\], locally deployable open\-source models are now widely used to answer medical and health questions\[[11](https://arxiv.org/html/2610.36513#bib.bib19),[12](https://arxiv.org/html/2610.36513#bib.bib18)\]\. Unlike ordinary question answering, a medical question is often less about factual correctness than about a trade\-off between competing values\[[13](https://arxiv.org/html/2610.36513#bib.bib21)\]\. Such ethical judgments have no single correct answer, yet they directly determine whether a model’s advice is safe and responsible\. The medical\-ethics behavior of these models therefore deserves systematic evaluation\.

While some work has begun to evaluate LLMs on medical tasks, it largely focuses on knowledge correctness\[[14](https://arxiv.org/html/2610.36513#bib.bib24),[15](https://arxiv.org/html/2610.36513#bib.bib22),[16](https://arxiv.org/html/2610.36513#bib.bib23)\]\. By contrast, the ethical behavior of these models has received far less attention\. The few studies that address it tend to treat ethics as an intrinsic property of the model, rather than as something shaped by the policy given at deployment\[[17](https://arxiv.org/html/2610.36513#bib.bib26),[18](https://arxiv.org/html/2610.36513#bib.bib25)\]\. Such evaluations also typically rely on a single LLM as the judge\[[18](https://arxiv.org/html/2610.36513#bib.bib25)\], which readily injects its own preferences into the scores and makes the conclusions hard to trust\[[19](https://arxiv.org/html/2610.36513#bib.bib27)\]\. In practice, however, a deployed model can rarely be retrained, and its ethical behavior is governed mainly by the policy written into the system prompt, commonly referred to as a*constitution*\[[20](https://arxiv.org/html/2610.36513#bib.bib28)\]\. This naturally raises our research question\.How does aconstitutionreshape a model’sbehavioracross ethical dimensions, does it benefit all models equally, and can it backfire?

To comprehensively evaluate the medical\-ethics behavior of locally deployable LLMs, we proposeMedConstBench, a medical\-ethics benchmark organized around Beauchamp’s four principles\[[21](https://arxiv.org/html/2610.36513#bib.bib16)\]\. At its core is MedDilemma, our dataset of clinical dilemmas with conflicting principles, which targets ethical decision\-making rather than factual recall\. To test the generalization of our findings, we additionally evaluate on the public MedEthicsQA\[[22](https://arxiv.org/html/2610.36513#bib.bib30)\], so that the study spans both clinical decision\-making and ethical discourse\. Figure[1](https://arxiv.org/html/2610.36513#S1.F1)illustrates our pipeline\. Each item is answered by a set of widely used open\-source models under three representative constitutions plus a baseline with no constitution\. Scoring open\-ended ethical responses is inherently subjective\[[23](https://arxiv.org/html/2610.36513#bib.bib29)\]\. To remove the bias of any single evaluator, MedConstBench adopts multi\-model peer review, in which each response is independently scored by the other models against the four principles \(autonomy, non\-maleficence, beneficence, justice\), and the ratings are aggregated into one score per principle\. Calibrated against a human gold standard, multi\-model peer review correlates with human judgment substantially better than a single judge, making it a trustworthy measurement instrument\.

![Refer to caption](https://arxiv.org/html/2610.36513v1/1027.png)Fig\. 1:Overview of theMedConstBenchpipeline, illustrated on an autonomy dilemma\.Step 1 \(Inputs\):each medical\-ethics itemq∈𝒬q\\in\\mathcal\{Q\}is presented to every evaluated model under each constitutioncc\.Step 2 \(Response and Judge\):each model produces a responset=\(m,c,q\)t=\(m,c,q\), which a leave\-one\-out panel of LLM judges scores against Beauchamp’s four principles \(xj,a​\(t\)∈\{0,1,2\}x\_\{j,a\}\(t\)\\in\\\{0,1,2\\\}\)\.Step 3 \(Performance Score\):ratings are aggregated across judges, within each principle, and across items into per\-principle scoresScoreP​\(m,c\)\\mathrm\{Score\}\_\{P\}\(m,c\)and an overall score, visualized as per\-constitution radar charts\.Step 4 \(Constitution Selection\):a do\-no\-harm gate keeps only admissible constitutions and selects the one maximizing the priority principleP⋆P^\{\\star\}\.Our experiments show that the effect of a constitution is dimension selective\. Different constitutions emphasize different principles, and they cannot all be optimized at the same time\. More critically, the effect is model dependent\. A constitution that is best for most models can collapse an individual model across the board\. We therefore propose a do\-no\-harm adaptive selector\. Within the feasible set in which no principle drops appreciably below the baseline, it picks the constitution best aligned with the deployment priority\. The selector matches each model’s oracle while guaranteeing a worst case floor, and it generalizes to unseen models at low deployment cost\.

Overall, our contributions are as follows\.

- •We propose MedConstBench, to the best of our knowledge the first medical\-ethics benchmark that treats the deployment constitution as a controlled variable\. It includes our own clinical\-dilemma dataset MedDilemma\.
- •We conduct a systematic evaluation of widely used open\-source LLMs under four representative constitutions, and find that the effect of constitutions is dimension selective and model dependent, with no universally optimal choice and a real risk of backfiring\.
- •We propose and validate a do\-no\-harm adaptive constitution selector that matches the per\-model oracle while guaranteeing a worst\-case lower bound, and generalizes to new models at low cost\.

## IIRelated Work

### II\-ABenchmarks for Medical Tasks

Evaluation of LLMs on medical tasks remains largely focused on knowledge correctness\. Multiple\-choice benchmarks such as MedQA\[[15](https://arxiv.org/html/2610.36513#bib.bib22)\], PubMedQA\[[14](https://arxiv.org/html/2610.36513#bib.bib24)\], and MedMCQA\[[16](https://arxiv.org/html/2610.36513#bib.bib23)\]assess clinical knowledge and reasoning, while HealthBench\[[24](https://arxiv.org/html/2610.36513#bib.bib31)\]and MultiMedQA\[[25](https://arxiv.org/html/2610.36513#bib.bib32)\]extend evaluation to open\-ended medical question answering\. Evaluation of medical ethics is far less common and varies in focus\. MedSafetyBench\[[26](https://arxiv.org/html/2610.36513#bib.bib33)\]tests whether a model can refuse unsafe or malicious requests, and MedEthicsQA\[[22](https://arxiv.org/html/2610.36513#bib.bib30)\]assesses normative ethical knowledge over abstract conflicts via large\-scale multiple\-choice questions\. PrinciplismQA\[[18](https://arxiv.org/html/2610.36513#bib.bib25)\]extends this to real clinical dilemmas with multiple clinically valid options, and MedEthicEval\[[27](https://arxiv.org/html/2610.36513#bib.bib34)\]distinguishes different types of ethical dilemmas in Chinese settings\. However, these benchmarks largely treat ethical behavior as an intrinsic property of the model, and typically rely on a single LLM judge, which is known to inject self\-preference bias\[[19](https://arxiv.org/html/2610.36513#bib.bib27),[23](https://arxiv.org/html/2610.36513#bib.bib29)\]\. MedConstBench addresses this gap by treating the constitution as a controlled variable and scoring responses with human\-calibrated multi\-model peer review\.

TABLE I:Comparison of the four constitutions by source and design orientation\.
### II\-BConstitutions\.

A constitution is an ethical policy written into the system prompt to steer a model’s behavior at deployment\. The core idea is to guide model behavior with a set of explicitly written principles rather than implicit preference data\[[20](https://arxiv.org/html/2610.36513#bib.bib28)\]\. Unlike retraining, a constitution requires no change to model weights\. This makes it the most direct and widely used lever once a model is deployed\. Existing work mostly treats the constitution as a training or alignment device\[[31](https://arxiv.org/html/2610.36513#bib.bib35)\], or studies steering only in general non\-medical settings\[[32](https://arxiv.org/html/2610.36513#bib.bib36),[33](https://arxiv.org/html/2610.36513#bib.bib37)\]\. It rarely treats the deployment\-time constitution as a controlled variable for examining how it reshapes a model’s behavior across ethical dimensions\. We study three representative medical\-ethics constitutions: AMA, a professional code of physician duties that foregrounds patient welfare\[[28](https://arxiv.org/html/2610.36513#bib.bib13)\]; WHO, a health\-AI governance framework stressing patient autonomy and equitable access\[[29](https://arxiv.org/html/2610.36513#bib.bib14)\]; and Mitigation, a simple “first, do no harm” safety prompt prioritizing risk avoidance\[[30](https://arxiv.org/html/2610.36513#bib.bib15)\]\.

## IIIMethodology

### III\-AProblem Setup

We consider a set of locally\-deployable LLMsℳ\\mathcal\{M\}evaluated on a collection of medical\-ethics items𝒬\\mathcal\{Q\}, where each itemq∈𝒬q\\in\\mathcal\{Q\}poses an ethically charged medical question or scenario\.

Prompting\.Each item is presented to every model under one of four representative constitutions𝒞=\{C0,C1,C2,C3\}\\mathcal\{C\}=\\\{C\_\{0\},C\_\{1\},C\_\{2\},C\_\{3\}\\\}, whereC0C\_\{0\}is the no\-constitution baseline against which all effects are measured\. These constitutions are chosen as typical ethical policies that each emphasize a different focus, as shown in Table[I](https://arxiv.org/html/2610.36513#S2.T1)\. Each response is therefore identified by a triple

t=\(m,c,q\)∈ℳ×𝒞×𝒬,t=\(m,c,q\)\\in\\mathcal\{M\}\\times\\mathcal\{C\}\\times\\mathcal\{Q\},i\.e\. the answer of modelmmto itemqqunder constitutioncc\.

TABLE II:Evaluation rubric: Beauchamp’s four principles operationalized as graded criteria\. Each criterion is scored 0 \(not met\), 1 \(partially met\), or 2 \(fully met\)\.Evaluation\.Each such responsettis assessed against Beauchamp’s four principles of biomedical ethics\[[21](https://arxiv.org/html/2610.36513#bib.bib16)\], collectively denoted𝒫\\mathcal\{P\}and comprising Autonomy, Non\-maleficence, Beneficence, and Justice\. Every principleP∈𝒫P\\in\\mathcal\{P\}is operationalized as a setAPA\_\{P\}of three binary\-graded criteria \(\|AP\|=3\|A\_\{P\}\|=3\), giving a total of twelve criteriaA=⋃P∈𝒫APA=\\bigcup\_\{P\\in\\mathcal\{P\}\}A\_\{P\}\(\|A\|=12\|A\|=12\); the full rubric is given in Table[II](https://arxiv.org/html/2610.36513#S3.T2)\. Each criteriona∈Aa\\in Ais scored on the ordinal scale\{0,1,2\}\\\{0,1,2\\\}\. A panel of judge models scorestton all twelve criteria, and the next section describes how these criterion\-level scores are aggregated into per\-principle and overall scores\.

### III\-BScoring via Multi\-Model Peer Review

To mitigate the self\-preference bias of single\-model evaluation\[[19](https://arxiv.org/html/2610.36513#bib.bib27),[23](https://arxiv.org/html/2610.36513#bib.bib29)\], we score each response by multi\-model peer review, in which a panelJJof judge models independently rates the response and the ratings are aggregated\. The model under evaluation is excluded from its own panel\.

For a responset=\(m,c,q\)t=\(m,c,q\), each judgej∈Jj\\in Janswers all twelve criteria\. For criterionaa, judgejjreturnsxj,a​\(t\)∈\{0,1,2\}x\_\{j,a\}\(t\)\\in\\\{0,1,2\\\}, where larger values indicate fuller satisfaction of that criterion:

xj,a​\(t\)=\{0,criterion not met, or contradicted,1,partially met \(mentioned but implicit\),2,fully met \(explicit and well\-grounded\)\.x\_\{j,a\}\(t\)=\\begin\{cases\}0,&\\text\{criterion not met, or contradicted,\}\\\\ 1,&\\text\{partially met \(mentioned but implicit\),\}\\\\ 2,&\\text\{fully met \(explicit and well\-grounded\)\.\}\\end\{cases\}
Ratings are aggregated in three stages\.

*\(i\) Across judges\.*At the criterion level we take an equal\-weight average over the panel,

s¯a​\(t\)=1\|J\|​∑j∈Jxj,a​\(t\)∈\[0,2\]\.\\bar\{s\}\_\{a\}\(t\)\\;=\\;\\frac\{1\}\{\|J\|\}\\sum\_\{j\\in J\}x\_\{j,a\}\(t\)\\;\\in\[0,2\]\.\(1\)
*\(ii\) Within a principle\.*The score of principlePPon responsettis the mean of its three criteria,

d^P​\(t\)=1\|AP\|​∑a∈APs¯a​\(t\)\.\\hat\{d\}\_\{P\}\(t\)\\;=\\;\\frac\{1\}\{\|A\_\{P\}\|\}\\sum\_\{a\\in A\_\{P\}\}\\bar\{s\}\_\{a\}\(t\)\.\(2\)
*\(iii\) Across items\.*Fixing a model–constitution pair\(m,c\)\(m,c\)and averaging over all itemsq∈𝒬q\\in\\mathcal\{Q\}yields the per\-principle scores

ScoreP​\(m,c\)=1\|𝒬\|​∑q∈𝒬d^P​\(\(,,,,,\)\),\\mathrm\{Score\}\_\{P\}\(m,c\)\\;=\\;\\frac\{1\}\{\|\\mathcal\{Q\}\|\}\\sum\_\{q\\in\\mathcal\{Q\}\}\\hat\{d\}\_\{P\}\\bigl\(\(m,c,q\)\\bigr\),\(3\)and the overall score is their mean across the four principles,

Overall⁡\(m,c\)=14​∑PScoreP​\(m,c\)\.\\mathrm\{Overall\}\(m,c\)\\;=\\;\\frac\{1\}\{4\}\\sum\_\{P\}\\mathrm\{Score\}\_\{P\}\(m,c\)\.\(4\)

### III\-CDo\-No\-Harm Adaptive Constitution Selection

Rather than applying one fixed constitution to all models, we select a constitution*per model*under a do\-no\-harm constraint\. Fix a modelmmand a deployment\-priority principleP⋆∈𝒫P^\{\\star\}\\in\\mathcal\{P\}chosen by the deployer\. We take the no\-constitution baselineC0C\_\{0\}as reference and a toleranceδ=0\.1\\delta=0\.1\.

A constitution is*admissible*formmif it lowers no principle below itsC0C\_\{0\}value by more thanδ\\delta\. The admissible set is

ℱ\(m\)=\{c∈𝒞:ΔP\(m,c\)≥−δ∀P\},\\mathcal\{F\}\(m\)\\;=\\;\\bigl\\\{\\,c\\in\\mathcal\{C\}\\;:\\;\\Delta\_\{P\}\(m,c\)\\geq\-\\delta\\;\\;\\forall P\\,\\bigr\\\},\(5\)whereΔP​\(m,c\)=ScoreP​\(m,c\)−ScoreP​\(m,C0\)\\Delta\_\{P\}\(m,c\)=\\mathrm\{Score\}\_\{P\}\(m,c\)\-\\mathrm\{Score\}\_\{P\}\(m,C\_\{0\}\)is the change in principlePPrelative to baseline\.

Within this admissible set we select the constitution that maximizes the priority principle:

c⋆​\(m,P⋆\)=arg​maxc∈ℱ⁡\(m\)⁡ScoreP⋆​\(m,c\)\.c^\{\\star\}\(m,P^\{\\star\}\)\\;=\\;\\operatorname\*\{arg\\,max\}\_\{c\\,\\in\\,\\mathcal\{F\}\(m\)\}\\;\\mathrm\{Score\}\_\{P^\{\\star\}\}\(m,c\)\.\(6\)SinceΔP​\(m,C0\)=0\\Delta\_\{P\}\(m,C\_\{0\}\)=0for allPP, the baselineC0C\_\{0\}always satisfies the constraint, soℱ⁡\(m\)\\mathcal\{F\}\(m\)is non\-empty\. In the worst case the selector returnsC0C\_\{0\}, so no principle drops more thanδ\\deltabelow baseline\.

## IVExperiments

### IV\-AExperimental Setup

Models\.We evaluate ten widely used open\-source LLMs that span different families, sizes, and release generations\. The full list of models is given in Table[III](https://arxiv.org/html/2610.36513#S4.T3)\. To test whether our findings generalize to unseen models, we additionally hold out three models \(Table[V](https://arxiv.org/html/2610.36513#S4.T5)\) that are never used when deriving the selection rule\. All models receive the same prompts and decoding settings, so that any difference in behavior is attributable to the constitution rather than to prompting\.

Datasets\.To evaluate medical\-ethics behavior under conflicting principles, we construct MedDilemma, our own dataset of 400 clinical dilemmas targeting ethical decision\-making\. We build it through an LLM\-assisted pipeline with human filtering\. Seeded with real cases from authoritative clinical\-ethics references\[[21](https://arxiv.org/html/2610.36513#bib.bib16),[34](https://arxiv.org/html/2610.36513#bib.bib38)\], we prompt Claude111https://www\.anthropic\.com/claudeto generate clinical dilemmas, and all generated items are then manually reviewed and filtered for clinical plausibility, principle conflict, and quality, yielding 400 dilemmas\. We further evaluate on MedEthicsQA\[[22](https://arxiv.org/html/2610.36513#bib.bib30)\], a public benchmark of ethics\-discourse questions, from which we sample 400 items to test generalization beyond our own data\. Together they cover both clinical decision\-making and ethical discourse\.

Constitutions\.Each item is evaluated under four constitutions𝒞=\{C0,C1,C2,C3\}\\mathcal\{C\}=\\\{C\_\{0\},C\_\{1\},C\_\{2\},C\_\{3\}\\\}\(None, AMA, WHO, Mitigation\), whereC0C\_\{0\}is the no\-constitution baseline\.

Judges and implementation\.The ten evaluated models also serve as the judge pool\. Under the leave\-one\-out protocol, each response is scored by the other nine \(\|J\|=9\|J\|=9\)\. For each criterion, judges return an ordinal score in\{0,1,2\}\\\{0,1,2\\\}following the rubric in Table[II](https://arxiv.org/html/2610.36513#S3.T2)\. We use the default temperature and identical decoding settings across all models\. In total this yields over 3\.4M criterion\-level judgments, spanning 10 models under 4 constitutions on 2 datasets, each response scored by 9 judges on 12 criteria\.

![Refer to caption](https://arxiv.org/html/2610.36513v1/fig2_single.png)Fig\. 2:Per\-principle change \(Δ\\Delta\) relative to the no\-constitution baselineC0C\_\{0\}under AMA, WHO, and Mitigation on MedEthicsQA and MedDilemma\. Blue indicates an increase over baseline and red a decrease\.
### IV\-BMain Results

We first measure how a constitution changes a model’s medical\-ethics behavior\. Table[III](https://arxiv.org/html/2610.36513#S4.T3)reports the scores of all ten models under the four constitutions on both datasets, and Figures[2](https://arxiv.org/html/2610.36513#S4.F2)and[3](https://arxiv.org/html/2610.36513#S4.F3)visualize these results as per\-principle changes relative to the baseline and as radar charts\. Three patterns stand out, organized by how the effect varies across principles, across models, and across datasets\.

TABLE III:Beauchamp four\-principle scores and overall mean of 10 LLMs under four constitutions on MedEthicsQA and MedDilemma\. For each model and dataset, the highest overall mean is inbold\.Effect across principles\.The four principles do not respond to constitutions in the same way \(Table[III](https://arxiv.org/html/2610.36513#S4.T3)\)\. Beneficence is already high at baseline, with a mean of1\.891\.89on MedDilemma, and barely changes under any constitution, whereas Autonomy and Justice start lower and move the most\. This is clearest in the per\-principle change relative to the baseline \(Figure[2](https://arxiv.org/html/2610.36513#S4.F2)\), where the Autonomy and Justice columns carry most of the color while Beneficence stays near zero\. The effect of a constitution is thus dimension\-selective, reshaping the value\-laden dimensions rather than the near\-saturated safety ones, so no constitution improves all principles at once\. The radar charts in Figure[3](https://arxiv.org/html/2610.36513#S4.F3)show the same shape, with Beneficence fully extended and the constitutions differing mainly along Autonomy and Justice\.

Effect across models\.Adding a constitution helps most models\. Relative to the baseline, the great majority of cells in Figure[2](https://arxiv.org/html/2610.36513#S4.F2)are blue, and every model has at least one constitution that improves its overall score\. This makes a fixed choice such as WHO look attractive, since WHO is optimal for 8 of 10 models on MedDilemma and 7 of 10 on MedEthicsQA\. For Llama\-3\-8B, however, WHO is catastrophic\. On MedDilemma it lowers the overall score from1\.571\.57to1\.251\.25and drives Justice to1\.041\.04, the lowest in the table\. The same WHO that raises MiMo\-7B across all principles turns deep red for Llama\-3\-8B\. The effect of a constitution therefore depends on the model it is applied to and cannot be read off the constitution alone, and a wrong choice can backfire\.

Effect across datasets and dimensions\.These effects are stronger on MedDilemma than on MedEthicsQA\. On MedEthicsQA scores stay in a narrow band and constitutions induce only mild shifts, whereas the conflicting\-principle dilemmas of MedDilemma spread the models apart and expose the backfiring on Llama\-3\-8B\. Clinical dilemmas are thus more discriminative than ethical discourse\. Across both datasets, Justice is the most sensitive dimension, with the lowest baseline, the widest spread, and the largest swings, and it is where Llama\-3\-8B collapses most\. The best constitution varies across both models and datasets, so no constitution is universally optimal\. Committing to any single one is suboptimal for some models and, as Llama\-3\-8B shows, can be actively harmful\.

Fig\. 3:Per\-principle scores of ten LLMs under the four constitutions on \(a\) MedEthicsQA and \(b\) MedDilemma\. Axes are Beauchamp’s four principles, namely Autonomy \(Auton\.\), Non\-maleficence \(Non\-mal\.\), Beneficence \(Benef\.\), and Justice, together with the Overall mean\. Axis labels are shown on the leftmost chart of each row\. The shaded area marks the no\-constitution baselineC0C\_\{0\}\. The radial axis spans\[1\.4,2\.0\]\[1\.4,2\.0\]in \(a\) and\[1\.0,2\.0\]\[1\.0,2\.0\]in \(b\)\.
### IV\-CDo\-No\-Harm Adaptive Selection

Since no constitution is universally optimal and a wrong choice can backfire, a fixed constitution is unsafe to deploy\. We therefore select a constitution per model under the do\-no\-harm gate defined in Section[III](https://arxiv.org/html/2610.36513#S3), which keeps only constitutions that lower no principle below the baseline and then picks the one best aligned with the deployment priority\. We evaluate it along three aspects, recovery, generalization, and cost\.

Adaptive selection matches the oracle\.Table[IV](https://arxiv.org/html/2610.36513#S4.T4)compares constitution\-selection strategies on MedDilemma by their overall and Justice scores\. Every fixed constitution has a clear weakness\. WHO is the sharpest example, with the best average yet the worst minimum, dragged down by Llama\-3\-8B to1\.251\.25in overall score and1\.041\.04in Justice\. The do\-no\-harm adaptive selector raises the worst\-case overall score to1\.671\.67and Justice to1\.371\.37, both higher than any fixed strategy, while also attaining the best average\. Its mean and worst\-case values are identical to the oracle\.

TABLE IV:Constitution\-selection strategies on MedDilemma: mean and worst\-case \(min over models\) Overall/Justice\. Adaptive = do\-no\-harm\-gated selector; best worst\-case inbold\.The rule generalizes to new models\.We test the do\-no\-harm selection strategy on three held\-out LLMs that are never used when deriving the rule\. Comparing its choice against each model’s per\-model oracle, we find that on all six model–dataset pairs \(Table[V](https://arxiv.org/html/2610.36513#S4.T5)\) the adaptive selection picks the same constitution as the oracle\. These held\-out models span different families, sizes, and generations, showing that the rule captures a transferable pattern rather than memorizing specific models\.

TABLE V:Generalization to different held\-out models \(Overall score\)\. MEQA = MedEthicsQA, MedDil = MedDilemma\.Low\-cost deployment\.Selecting a constitution for a new model would in principle require evaluating it on all 400 items, but in practice far fewer are needed\. Table[VI](https://arxiv.org/html/2610.36513#S4.T6)uses aKK\-item probe to make the selection and compares it against the full 400\-item evaluation\. With only 30 items, the probe agrees with the full evaluation about 90% of the time, and the worst\-case overall score lost relative to the oracle is below0\.020\.02\. Deploying the adaptive selector to a new model is therefore cheap and does not require rerunning the full benchmark\.

TABLE VI:Probe sufficiency on MedDilemma: aKK\-item probe to select the constitution vs\. all 400 items\. Agreement, mean and worst\-case regret \(Overall score lost vs\. oracle\)\.
### IV\-DHuman Validation

Since the quality of the scores directly affects the credibility of all our conclusions, we validate the reliability of multi\-model peer review\. We ask three annotators to independently score 100 items and take the mean of the three as the human gold standard, then check the protocol from two angles, whether the judges agree with one another and whether their scores align with this gold standard\.

The judges agree closely\.Table[VII](https://arxiv.org/html/2610.36513#S4.T7)reports the quadratic\-weighted Cohen’sκ\\kappaamong the judges on the four principles\. All values fall between0\.620\.62and0\.700\.70, with an average of0\.670\.67, in the substantial\-agreement range\[[47](https://arxiv.org/html/2610.36513#bib.bib39)\]\. Agreement is lowest on Justice \(0\.620\.62\), consistent with our main results where Justice is the most sensitive and hardest principle to judge\. Overall, different judges score the same response consistently, showing that the scores are not random noise\.

TABLE VII:Inter\-judge agreement across the four bioethical principles \(quadratic\-weighted Cohen’sκ\\kappa\)\.Multi\-judge scores align with human judgment\.Table[VIII](https://arxiv.org/html/2610.36513#S4.T8)compares the single\-judge and multi\-judge protocols against the gold standard\. A single judge reaches only a Spearmanρ\\rhoof0\.590\.59and a mean absolute error of0\.400\.40\. Multi\-model peer review raisesρ\\rhoto0\.760\.76and lowers the error to0\.260\.26, substantially better than a single judge and close to the inter\-annotator ceiling of0\.790\.79\. This shows that multi\-model peer review reduces the bias of a single judge and serves as a human\-aligned, reliable scoring protocol\.

TABLE VIII:Alignment with the human gold standard\. Spearman’sρ\\rhomeasures ranking; MAE measures calibration\. Inter\-annotator agreement provides the ceiling\.

## VConclusion

We introduced MedConstBench, a medical\-ethics benchmark that treats the deployment constitution as a controlled variable and scores responses with human\-calibrated multi\-model peer review\. Across ten open\-source LLMs on both our MedDilemma dataset and the public MedEthicsQA, we found that the effect of a constitution is dimension\-selective and model\-dependent\. Constitutions mainly reshape the value\-laden dimensions, and a constitution that helps most models can degrade an individual one\. Because no constitution is universally optimal, we proposed a do\-no\-harm adaptive selector that matches the per\-model oracle, guarantees a worst\-case floor, generalizes to unseen models, and deploys at low cost\. Our results suggest that the ethical policy supplied at deployment deserves to be evaluated and chosen as carefully as the model itself\. In future work, we plan to extend the benchmark to more constitutions and biomedical domains\.

## References

- \[1\]Z\. Wang, J\. Hu, and R\. Mu\(2025\)Safety of embodied navigation: a survey\.External Links:2508\.05855Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p1.1)\.
- \[2\]Y\. Yang\(2025\)CLMN: concept based language models via neural symbolic reasoning\.In2025 IEEE International Conference on Systems, Man, and Cybernetics \(SMC\),pp\. 795–801\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p1.1)\.
- \[3\]Y\. Yang, F\. Lei, Y\. Sun, Y\. Zeng, C\. Lv, J\. Hong, J\. Tian, T\. Qiu, X\. Wang, Y\. Chen,et al\.\(2026\)AIDABench: ai data analytics benchmark\.arXiv preprint arXiv:2603\.15636\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p1.1)\.
- \[4\]F\. Lei, Y\. Yang, W\. Sun, and D\. Lin\(2025\)Mcpverse: an expansive, real\-world benchmark for agentic tool use\.arXiv preprint arXiv:2508\.16260\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p1.1)\.
- \[5\]Z\. Wang, G\. Jin, J\. Hu, and R\. Mu\(2026\)Clucert: certifying llm robustness via clustering\-guided denoising smoothing\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 37998–38006\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p1.1)\.
- \[6\]Z\. A\. Nazi and W\. Peng\(2024\)Large language models in healthcare and medical domain: a review\.InInformatics,Vol\.11,pp\. 57\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p1.1)\.
- \[7\]Y\. Yang, Z\. Huang, H\. Yang, and Z\. Wang\(2026\)DeliMF: gated cross\-attention fusion of structured time series and clinical text for early intensive care delirium prediction\.arXiv preprint arXiv:2610\.24585\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p1.1)\.
- \[8\]P\. Guo, P\. Tewari, E\. L\. W\. Choo, K\. B\. Tan, J\. Abisheganaden, and B\. Dickens\(2024\)Forecasting infectious and parasitic disease emergency department attendances using high\-dimensional time series data\.In2024 IEEE Conference on Artificial Intelligence \(CAI\),pp\. 603–610\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p1.1)\.
- \[9\]P\. Guo, J\. T\. Lim, L\. E\. Wee, A\. T\. Tay, Y\. Zhai, C\. J\. Chiew, B\. Ong, D\. C\. B\. Lye, and K\. B\. Tan\(2026\)Characterization of post\-acute multi\-organ sequelae following sars\-cov\-2 infection in the delta and omicron eras in a highly boosted population\.BMC Global and Public Health4\(1\),pp\. 15\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p1.1)\.
- \[10\]P\. Guo, W\. Y\. Loh, K\. Maung, B\. L\. Dickens, S\. Pei, E\. L\. W\. Choo, K\. B\. Tan, J\. Abishgenadan, P\. Ma, and J\. T\. Lim\(2026\)Ascertaining cause\-specific emergency department demand using forecast combinations\.BMC Emergency Medicine26\(1\),pp\. 83\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p1.1)\.
- \[11\]G\. Peikos, P\. Kasela, and G\. Pasi\(2024\)Leveraging large language models for medical information extraction and query generation\.In2024 IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology \(WI\-IAT\),pp\. 367–372\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p1.1)\.
- \[12\]D\. Lee, A\. Vaid, K\. M\. Menon, R\. Freeman, D\. S\. Matteson, M\. L\. Marin, G\. N\. Nadkarni,et al\.\(2025\)Using large language models to automate data extraction from surgical pathology reports: retrospective cohort study\.JMIR Formative Research9\(1\),pp\. e64544\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p1.1)\.
- \[13\]J\. C\. L\. Ong, S\. Y\. Chang, W\. William, A\. J\. Butte, N\. H\. Shah, L\. S\. T\. Chew, N\. Liu, F\. Doshi\-Velez, W\. Lu, J\. Savulescu,et al\.\(2024\)Medical ethics of large language models in medicine\.NEJM AI1\(7\),pp\. AIra2400038\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p1.1)\.
- \[14\]Q\. Jin, B\. Dhingra, Z\. Liu, W\. Cohen, and X\. Lu\(2019\)Pubmedqa: a dataset for biomedical research question answering\.InProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing \(EMNLP\-IJCNLP\),pp\. 2567–2577\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p2.1),[§II\-A](https://arxiv.org/html/2610.36513#S2.SS1.p1.1)\.
- \[15\]D\. Jin, E\. Pan, N\. Oufattole, W\. Weng, H\. Fang, and P\. Szolovits\(2021\)What disease does this patient have? a large\-scale open domain question answering dataset from medical exams\.Applied Sciences11\(14\),pp\. 6421\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p2.1),[§II\-A](https://arxiv.org/html/2610.36513#S2.SS1.p1.1)\.
- \[16\]A\. Pal, L\. K\. Umapathi, and M\. Sankarasubbu\(2022\)Medmcqa: a large\-scale multi\-subject multi\-choice dataset for medical domain question answering\.InConference on health, inference, and learning,pp\. 248–260\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p2.1),[§II\-A](https://arxiv.org/html/2610.36513#S2.SS1.p1.1)\.
- \[17\]J\. Jiao, S\. Afroogh, A\. Murali, K\. Chen, D\. Atkinson, and A\. Dhurandhar\(2025\)LLM ethics benchmark: a three\-dimensional assessment system for evaluating moral reasoning in large language models\.External Links:2505\.00853Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p2.1)\.
- \[18\]C\. Hong, M\. Wu, Q\. Xiao, Y\. Wang, X\. Wan, G\. Yu, B\. Wang, and Y\. Hu\(2026\)PrinciplismQA: a philosophy\-grounded approach to assessing llm\-human clinical medical ethics alignment\.External Links:2508\.05132,[Link](https://arxiv.org/abs/2508.05132)Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p2.1),[§II\-A](https://arxiv.org/html/2610.36513#S2.SS1.p1.1)\.
- \[19\]L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. Xing,et al\.\(2023\)Judging llm\-as\-a\-judge with mt\-bench and chatbot arena\.Advances in neural information processing systems36,pp\. 46595–46623\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p2.1),[§II\-A](https://arxiv.org/html/2610.36513#S2.SS1.p1.1),[§III\-B](https://arxiv.org/html/2610.36513#S3.SS2.p1.1)\.
- \[20\]Y\. Bai, S\. Kadavath, S\. Kundu, A\. Askell, J\. Kernion, A\. Jones, A\. Chen, A\. Goldie, A\. Mirhoseini, C\. McKinnon,et al\.\(2022\)Constitutional ai: harmlessness from ai feedback\.arXiv preprint arXiv:2212\.08073\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p2.1),[§II\-B](https://arxiv.org/html/2610.36513#S2.SS2.p1.1)\.
- \[21\]T\. Beauchamp and J\. Childress\(2019\)Principles of biomedical ethics: marking its fortieth anniversary\.Vol\.19,Taylor & Francis\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p3.1),[§III\-A](https://arxiv.org/html/2610.36513#S3.SS1.p3.1),[§IV\-A](https://arxiv.org/html/2610.36513#S4.SS1.p2.1)\.
- \[22\]J\. Wei, Z\. Meng, Z\. Xiao, T\. Hu, Y\. Feng, Z\. Zhou, J\. Wu, and Z\. Liu\(2025\)Medethicsqa: a comprehensive question answering benchmark for medical ethics evaluation of llms\.arXiv preprint arXiv:2506\.22808\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p3.1),[§II\-A](https://arxiv.org/html/2610.36513#S2.SS1.p1.1),[§IV\-A](https://arxiv.org/html/2610.36513#S4.SS1.p2.1)\.
- \[23\]K\. Wataoka, T\. Takahashi, and R\. Ri\(2024\)Self\-preference bias in llm\-as\-a\-judge\.arXiv preprint arXiv:2410\.21819\.Cited by:[§I](https://arxiv.org/html/2610.36513#S1.p3.1),[§II\-A](https://arxiv.org/html/2610.36513#S2.SS1.p1.1),[§III\-B](https://arxiv.org/html/2610.36513#S3.SS2.p1.1)\.
- \[24\]R\. K\. Arora, J\. Wei, R\. S\. Hicks, P\. Bowman, J\. Quiñonero\-Candela, F\. Tsimpourlas, M\. Sharman, M\. Shah, A\. Vallone, A\. Beutel,et al\.\(2025\)Healthbench: evaluating large language models towards improved human health\.arXiv preprint arXiv:2505\.08775\.Cited by:[§II\-A](https://arxiv.org/html/2610.36513#S2.SS1.p1.1)\.
- \[25\]K\. Singhal, S\. Azizi, T\. Tu, S\. S\. Mahdavi, J\. Wei, H\. W\. Chung, N\. Scales, A\. Tanwani, H\. Cole\-Lewis, S\. Pfohl,et al\.\(2023\)Large language models encode clinical knowledge\.Nature620\(7972\),pp\. 172–180\.Cited by:[§II\-A](https://arxiv.org/html/2610.36513#S2.SS1.p1.1)\.
- \[26\]T\. Han, A\. Kumar, C\. Agarwal, and H\. Lakkaraju\(2024\)Medsafetybench: evaluating and improving the medical safety of large language models\.Advances in neural information processing systems37,pp\. 33423–33454\.Cited by:[§II\-A](https://arxiv.org/html/2610.36513#S2.SS1.p1.1)\.
- \[27\]H\. Jin, J\. Shi, H\. Xu, K\. Zhu, and M\. Wu\(2025\)MedEthicEval: evaluating large language models based on chinese medical ethics\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 3: Industry Track\),pp\. 404–421\.Cited by:[§II\-A](https://arxiv.org/html/2610.36513#S2.SS1.p1.1)\.
- \[28\]F\. A\. Riddick\(2003\)The code of medical ethics of the american medical association\.Vol\.5,Ochsner Journal\.Cited by:[§II\-B](https://arxiv.org/html/2610.36513#S2.SS2.p1.1),[TABLE I](https://arxiv.org/html/2610.36513#S2.T1.4.3.2.1.1)\.
- \[29\]W\. Guidance\(2021\)Ethics and governance of artificial intelligence for health\.World Health Organization,pp\. 1–165\.Cited by:[§II\-B](https://arxiv.org/html/2610.36513#S2.SS2.p1.1),[TABLE I](https://arxiv.org/html/2610.36513#S2.T1.4.4.2.1.1)\.
- \[30\]M\. Omar, R\. Agbareia, J\. McGreevy, A\. Gorenshtein, A\. W\. Charney, A\. Sakhuja, B\. S\. Glicksberg, G\. N\. Nadkarni, and E\. Klang\(2025\)LLMs can do medical harm: stress\-testing clinical decisions under social pressure\.medRxiv,pp\. 2025–11\.Cited by:[§II\-B](https://arxiv.org/html/2610.36513#S2.SS2.p1.1),[TABLE I](https://arxiv.org/html/2610.36513#S2.T1.4.5.2.1.1)\.
- \[31\]S\. Kundu, Y\. Bai, S\. Kadavath, A\. Askell, A\. Callahan, A\. Chen, A\. Goldie, A\. Balwit, A\. Mirhoseini, B\. McLean,et al\.\(2023\)Specific versus general principles for constitutional ai\.arXiv preprint arXiv:2310\.13798\.Cited by:[§II\-B](https://arxiv.org/html/2610.36513#S2.SS2.p1.1)\.
- \[32\]W\. Zhao, Y\. Hu, Y\. Deng, J\. Guo, X\. Sui, X\. Han, A\. Zhang, Y\. Zhao, B\. Qin, T\. Chua,et al\.\(2025\)Beware of your po\! measuring and mitigating ai safety risks in role\-play fine\-tuning of llms\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 11112–11137\.Cited by:[§II\-B](https://arxiv.org/html/2610.36513#S2.SS2.p1.1)\.
- \[33\]K\. Samway, M\. N\. Takagi, R\. Mihalcea, B\. Schölkopf, I\. Chalkidis, D\. Hershcovich, and Z\. Jin\(2026\)When do language models endorse limitations on human rights principles?\.InFindings of the Association for Computational Linguistics: EACL 2026,pp\. 6597–6623\.Cited by:[§II\-B](https://arxiv.org/html/2610.36513#S2.SS2.p1.1)\.
- \[34\]A\. R\. Jonsen, M\. Siegler, and W\. J\. Winslade\(2010\)Clinical ethics: a practical approach to ethical decisions in clinical medicine\.Linacre Q50\(2\),pp\. 13\.Cited by:[§IV\-A](https://arxiv.org/html/2610.36513#S4.SS1.p2.1)\.
- \[35\]A\. Kamath, J\. Ferret, S\. Pathak, N\. Vieillard, R\. Merhej, S\. Perrin, T\. Matejovicova, A\. Ramé, M\. Rivière, L\. Rouillard,et al\.\(2025\)Gemma 3 technical report\.arXiv preprint arXiv:2503\.197864\.Cited by:[TABLE III](https://arxiv.org/html/2610.36513#S4.T3.5.3.1.1)\.
- \[36\]A\. Liu, B\. Feng, B\. Wang, B\. Wang, B\. Liu, C\. Zhao, C\. Dengr, C\. Ruan, D\. Dai, D\. Guo,et al\.\(2024\)Deepseek\-v2: a strong, economical, and efficient mixture\-of\-experts language model\.arXiv preprint arXiv:2405\.04434\.Cited by:[TABLE III](https://arxiv.org/html/2610.36513#S4.T3.5.7.1.1)\.
- \[37\]A\. Young, B\. Chen, C\. Li, C\. Huang, G\. Zhang, G\. Zhang, G\. Wang, H\. Li, J\. Zhu, J\. Chen,et al\.\(2024\)Yi: open foundation models by 01\. ai\.arXiv preprint arXiv:2403\.04652\.Cited by:[TABLE III](https://arxiv.org/html/2610.36513#S4.T3.5.11.1.1)\.
- \[38\]A\. H\. Liu, K\. Khandelwal, S\. Subramanian, V\. Jouault, A\. Rastogi, A\. Sadé, A\. Jeffares, A\. Jiang, A\. Cahill, A\. Gavaudan,et al\.\(2026\)Ministral 3\.arXiv preprint arXiv:2601\.08584\.Cited by:[TABLE III](https://arxiv.org/html/2610.36513#S4.T3.5.15.1.1)\.
- \[39\]M\. Abdin, J\. Aneja, H\. Behl, S\. Bubeck, R\. Eldan, S\. Gunasekar, M\. Harrison, R\. J\. Hewett, M\. Javaheripi, P\. Kauffmann,et al\.\(2024\)Phi\-4 technical report\.arXiv preprint arXiv:2412\.08905\.Cited by:[TABLE III](https://arxiv.org/html/2610.36513#S4.T3.5.19.1.1)\.
- \[40\]A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[TABLE III](https://arxiv.org/html/2610.36513#S4.T3.5.23.1.1)\.
- \[41\]B\. Hui, J\. Yang, Z\. Cui, J\. Yang, D\. Liu, L\. Zhang, T\. Liu, J\. Zhang, B\. Yu, K\. Lu,et al\.\(2024\)Qwen2\. 5\-coder technical report\.arXiv preprint arXiv:2409\.12186\.Cited by:[TABLE III](https://arxiv.org/html/2610.36513#S4.T3.5.27.1.1),[TABLE V](https://arxiv.org/html/2610.36513#S4.T5.4.2.1.1)\.
- \[42\]L\. Xiaomi, B\. Xia, B\. Shen, D\. Zhu, D\. Zhang, G\. Wang, H\. Zhang, H\. Liu, J\. Xiao, J\. Dong,et al\.\(2025\)MiMo: unlocking the reasoning potential of language model–from pretraining to posttraining\.arXiv preprint arXiv:2505\.07608\.Cited by:[TABLE III](https://arxiv.org/html/2610.36513#S4.T3.5.31.1.1)\.
- \[43\]T\. Glm, A\. Zeng, B\. Xu, B\. Wang, C\. Zhang, D\. Yin, D\. Zhang, D\. Rojas, G\. Feng, H\. Zhao,et al\.\(2024\)Chatglm: a family of large language models from glm\-130b to glm\-4 all tools\.arXiv preprint arXiv:2406\.12793\.Cited by:[TABLE III](https://arxiv.org/html/2610.36513#S4.T3.5.35.1.1)\.
- \[44\]T\. OLMo, P\. Walsh, L\. Soldaini, D\. Groeneveld, K\. Lo, S\. Arora, A\. Bhagia, Y\. Gu, S\. Huang, M\. Jordan,et al\.\(2024\)2 olmo 2 furious\.arXiv preprint arXiv:2501\.00656\.Cited by:[TABLE III](https://arxiv.org/html/2610.36513#S4.T3.5.39.1.1)\.
- \[45\]S\. Agarwal, L\. Ahmad, J\. Ai, S\. Altman, A\. Applebaum, E\. Arbus, R\. K\. Arora, Y\. Bai, B\. Baker, H\. Bao,et al\.\(2025\)Gpt\-oss\-120b & gpt\-oss\-20b model card\.arXiv preprint arXiv:2508\.10925\.Cited by:[TABLE V](https://arxiv.org/html/2610.36513#S4.T5.4.4.1.1)\.
- \[46\]A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[TABLE V](https://arxiv.org/html/2610.36513#S4.T5.4.6.1.1)\.
- \[47\]J\. R\. Landis and G\. G\. Koch\(1977\)The measurement of observer agreement for categorical data\.biometrics,pp\. 159–174\.Cited by:[§IV\-D](https://arxiv.org/html/2610.36513#S4.SS4.p2.1)\.