ConRub-Med: Reinforcement Learning with Consensus Rubrics for Open-Ended Medical Question Answering

arXiv cs.CL Papers
reinforcement-learning medical-question-answering rubrics grpo consensus open-ended healthbench

Summary

This paper introduces ConRub-Med, a reinforcement learning approach that uses consensus rubrics from multiple language models to reward open-ended medical question answering, achieving state-of-the-art results on several benchmarks including HealthBench-Hard.

arXiv:2608.10996v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards has been especially effective in mathematics and coding, where answers can be checked automatically. Many open-ended medical questions lack comparably cheap outcome verifiers: responses may be partly correct, incomplete, or contain clinically consequential errors. Rubrics written or validated by physicians offer strong clinical grounding, but involving experts in every instance is costly. Model-generated rubrics make this supervision scalable. We introduce ConRub-Med to preserve useful distinctions as rubric feedback moves from construction to policy optimization. For each prompt, three heterogeneous language models propose atomic criteria independently; a separate model reviews them, retaining only criteria with semantic support from all three generators. Three-State scoring distinguishes correct coverage, missing information, and incorrect claims. Errors receive negative rather than zero credit. When every response in a complete Group Relative Policy Optimization (GRPO) group receives the same final reward, a pairwise judge provides sequence advantages only if both candidate orders agree, without changing the scalar rewards. Groups without ties use vanilla GRPO. In a blinded study matched by question, two medical experts rate panels from the full pipeline as more clinically relevant than panels produced by one generator. Across the evaluated open models, ConRub-Med ranks first on six of nine benchmarks and achieves the highest medical and generalization averages. Using the resulting rubric dataset of 5,166 prompts, it scores $38.98 \pm 1.04$ (mean $\pm$ SD) on HealthBench-Hard, compared with InfiMed-ORBIT's 33.60 with 8,000 samples and 37.30 with 28,000.
Original Article
View Cached Full Text

Cached at: 08/12/26, 08:38 AM 

# ConRub-Med: Reinforcement Learning with Consensus Rubrics for Open-Ended Medical Question Answering
Source: [https://arxiv.org/abs/2608.10996](https://arxiv.org/abs/2608.10996)
Authors:[Taojie Zhu](https://arxiv.org/search/cs?searchtype=author&query=Zhu,+T),[Yuan Xia](https://arxiv.org/search/cs?searchtype=author&query=Xia,+Y),[Tao Sun](https://arxiv.org/search/cs?searchtype=author&query=Sun,+T),[Yizhi Wang](https://arxiv.org/search/cs?searchtype=author&query=Wang,+Y),[Yan Chen](https://arxiv.org/search/cs?searchtype=author&query=Chen,+Y),[Qunshan He](https://arxiv.org/search/cs?searchtype=author&query=He,+Q),[Tian Guan](https://arxiv.org/search/cs?searchtype=author&query=Guan,+T),[Jian Wang](https://arxiv.org/search/cs?searchtype=author&query=Wang,+J),[Jinjie Gu](https://arxiv.org/search/cs?searchtype=author&query=Gu,+J),[Junwei Liu](https://arxiv.org/search/cs?searchtype=author&query=Liu,+J),[Yonghong He](https://arxiv.org/search/cs?searchtype=author&query=He,+Y)

[View PDF](https://arxiv.org/pdf/2608.10996)

> Abstract:Reinforcement learning with verifiable rewards has been especially effective in mathematics and coding, where answers can be checked automatically\. Many open\-ended medical questions lack comparably cheap outcome verifiers: responses may be partly correct, incomplete, or contain clinically consequential errors\. Rubrics written or validated by physicians offer strong clinical grounding, but involving experts in every instance is costly\. Model\-generated rubrics make this supervision scalable\. We introduce ConRub\-Med to preserve useful distinctions as rubric feedback moves from construction to policy optimization\. For each prompt, three heterogeneous language models propose atomic criteria independently; a separate model reviews them, retaining only criteria with semantic support from all three generators\. Three\-State scoring distinguishes correct coverage, missing information, and incorrect claims\. Errors receive negative rather than zero credit\. When every response in a complete Group Relative Policy Optimization \(GRPO\) group receives the same final reward, a pairwise judge provides sequence advantages only if both candidate orders agree, without changing the scalar rewards\. Groups without ties use vanilla GRPO\. In a blinded study matched by question, two medical experts rate panels from the full pipeline as more clinically relevant than panels produced by one generator\. Across the evaluated open models, ConRub\-Med ranks first on six of nine benchmarks and achieves the highest medical and generalization averages\. Using the resulting rubric dataset of 5,166 prompts, it scores $38\.98 \\pm 1\.04$ \(mean $\\pm$ SD\) on HealthBench\-Hard, compared with InfiMed\-ORBIT's 33\.60 with 8,000 samples and 37\.30 with 28,000\.

## Submission history

From: Yan Chen \[[view email](https://arxiv.org/show-email/cb138667/2608.10996)\] **\[v1\]**Tue, 11 Aug 2026 14:48:15 UTC \(4,089 KB\)

Similar Articles

ARES: Automated Rubric Synthesis for Scalable LLM Reinforcement Learning

arXiv cs.CL

ARES proposes a framework for automatically constructing rubric-based RL data from pretraining documents, generating question-answer pairs and weighted rubrics to enable instance-level reward supervision for open-ended LLM responses, outperforming existing methods on multi-dimensional open-ended tasks.