Reinforcement Learning for Evidence-Seeking Diagnostic Reasoning with Large Language Models

# Reinforcement Learning for Evidence-Seeking Diagnostic Reasoning with Large Language Models
Source: [https://arxiv.org/html/2607.02983](https://arxiv.org/html/2607.02983)
Table 3:Agreement between human experts and LLM judges measured by Cohen’sκ\\kappa\.
### 5\.2Comparative Performance of Evidence\-seeking Diagnosis

Tables[5\.1](https://arxiv.org/html/2607.02983#S5.SS1)and[5\.1](https://arxiv.org/html/2607.02983#S5.SS1)summarize the performance of our framework against larger models and specialized medical reasoning baselines\. We performed bootstrap resampling 2,000 times and computed 95% confidence intervals for the overall performance to provide an uncertainty estimate\. The results reveal several key insights regarding model efficiency and reasoning alignment\. First, our Ours\-RL\-7B model exhibits remarkable performance, often eclipsing models with significantly larger parameter counts and specific data accumulation \(e\.g\., Qwen2\.5\-32B and Huatuo\-7B\)\. In the initial consultation stage, Ours\-RL\-7B achieves a leading DiffAcc of 62\.0 on the English dataset and 64\.8 on the Chinese dataset, outperforming the baselines\. This suggests that specialized RLVR can distill high\-level clinical reasoning, effectively narrowing the gap between small\-scale and frontier models\. While larger SFT models \(Ours\-SFT\-32B\) demonstrate robust pattern\-matching capabilities, our RL\-trained model shows superior strategic behavior in multilingual contexts\. In the follow\-up stage \(Table[5\.1](https://arxiv.org/html/2607.02983#S5.SS1)\), we observe that while our models maintain a strong lead over standard medical LLMs, there remains a performance ceiling when compared to massive reasoning models like QwQ\-32B and larger models like Qwen2\.5\-72B\. We attribute this to the complexity of the negative\-evidence reasoning \(the rule\-out process\), which requires a high degree of logical rigor and domain knowledge to synthesize additional IHC signals\. Meanwhile, we can observe substantial agreement \(\>0\.6\>0\.6\) between human experts and independent LLM judges, supporting the reliability of the evaluation protocol as shown in Table[3](https://arxiv.org/html/2607.02983#S5.T3)\. Agreement is slightly lower for DxAcc than DiffAcc, likely because final diagnosis evaluation requires finer\-grained distinctions among closely related disease entities\.

### 5\.3The Fidelity of Clinical Inquiry

A diagnostic model is only as reliable as the questions it asks\. As shown in Fig\.[4](https://arxiv.org/html/2607.02983#S5.F4), we evaluate the Clinical Plausibility of the requested examinations\. We use GPT\-5\-Minimal as an external evaluator\. GPT\-5 is tasked with assessing whether the suggested examination items are appropriate, given the model’s differential diagnosis\. To ensure stability and reduce variance, each evaluation is repeated three times, and the pass rate is reported as the final metric\. The moderate overall pass rates in current LLMs often stem from a thoroughness bias, where models request redundant examinations to verify each candidate in a broad differential list\. Our SFT and RL models demonstrate superior alignment with expert\-level examination protocols compared to their general\-purpose baselines, reflecting a successful transition to a hypothesis\-driven evidence\-seeking mode\.

### 5\.4Ablation Study

The ablation study investigates two key aspects: \(1\) the effectiveness of incorporating a rank\-sensitive diagnosis reward with differentτ\\tauvalues during RL \(RL reward ablation\), and \(2\) the contribution of the RAGES components \(RAGES ablation\)\.

Table 4:Ablation study on different diagnostic rewards\.
Table 5:Ablation study on different phases of RAGES\.

RL reward ablation\.We evaluate four variants of the diagnosis reward\. \(1\) Binary reward: where a hit receives 1 and otherwise 0, which can be seen as a limiting case whenτ→0\\tau\\to 0\. \(2\) Rank\-sensitive reward withτ=0\.8\\tau=0\.8and \(3\) withτ=1\.5\\tau=1\.5\. \(4\) Rank\-sensitive reward with dynamicτ\\tauvalues as the main setting\. We assess performance using three metrics: differential accuracy \(DiffAcc\), final diagnosis accuracy \(DxAcc\), and the average length of initial differential lists \(\#DDx\) to directly show how the reward function shapes the model’s diagnostic behavior\. A lowerτ\\taueffectively incentivizes a broader differential search, maximizing exploratory capability in the early stages\. Conversely, a higherτ\\tauenforces clinical focus, compelling the model to commit to the most likely diagnoses\. Our dynamicτ\\taustrategy outperforms all static configurations by effectively managing the uncertainty\-precision trade\-off\. It allows the model to be exploratory with sparse information and decisive once evidence arrives, mirroring the cognitive shift of human experts\.

RAGES ablation\.The efficacy of RAGES is validated in Table[5](https://arxiv.org/html/2607.02983#S5.T5)\. We observe that the co\-effect between faithful recitation and empirical knowledge is essential for closing the reasoning loop\. The full configuration achieves the highest correctness \(84\.8% overall\), confirming that grounding the simulator in a structured medical environment is superior to relying solely on an LLM’s raw internal knowledge\. This reinforces RAGES’s role as a reliable Clinical Oracle for training and evaluating autonomous evidence\-seeking models\.

## 6Conclusion and Discussion

In this work, we have formalized and addressed a fundamental gap in current medical LLMs: the transition from passive inference to autonomous diagnostic agency\. By framing diagnosis as an iterative evidence\-seeking trajectory, we demonstrate that models can be trained to resolve clinical ambiguity through strategic information acquisition\. Our framework, comprising the RLVR\-based reasoning engine, the RAGES clinical oracle, and a curated pathological corpus, provides a robust blueprint for closed\-loop medical reasoning\. Empirical results show that providing a mechanism for active inquiry leads to a quantum leap in diagnostic fidelity, with our RL\-trained models exhibiting a high degree of clinical priority and conciseness, even at a 7B\-parameter scale\.

Despite these advancements, our work marks the beginning of a broader research agenda\. Several high\-impact directions remain\. \(1\) While validated clinically, our rank\-sensitive rewardRdR\_\{d\}is domain\-agnostic\. It is mathematically applicable to any hierarchical decision\-making task requiring hypothesis prioritization\. Future work will explore how this reward shapes exploration\-to\-decision transitions in broader non\-medical contexts\. \(2\) Though exploited as an offline simulator, RAGES could be integrated into online RL loops to enable multi\-turn autonomous rollouts and long\-term diagnostic look\-ahead\. Furthermore, expanding the framework to include radiology and oncology data will test its robustness in complex, multi\-source informational environments\. \(3\) Future alignment should shift from outcome\-only metrics to process\-based reward design\. By aligning internal reasoning steps with established clinical guidelines, we can ensure the model reaches the correct diagnosis through a verifiable and trustworthy path\.

Overall, this work establishes a structured framework for interactive AI\-assisted diagnosis\. By empowering LLMs with the capacity for active information acquisition, we move closer to a future where AI serves not merely as a static encyclopedia but as a dynamic, evidence\-driven partner in the clinical diagnostic process\.

## Ethics Statement

This work leverages pathological case studies from multiple sources, raising two primary ethical considerations, i\.e\., patient privacy and data re\-distribution\.

Regarding patient privacy, all case reports collected from public websites and journals were already anonymized at the source\. We carefully removed any personally identifiable information from the in\-house dataset, preserving only essential information such as age and gender for clinical reasoning\.

Regarding data distribution, we will not release our in\-house dataset publicly due to institutional data protection policies\. For externally sourced cases, we strictly adhered to the usage guidelines specified by each website or journal\. To avoid unauthorized redistribution, we will release only the URLs linking to the original case sources, allowing other researchers to access the materials while respecting the original data ownership and licensing terms\.

The risk of this work may also lie in improper responses \(including repetitive patterns, false information, and offensive output\) since we do not specifically strengthen the safety of the model\. The presented cases may contain content that is offensive\.

## References

- Self\-evolving multi\-agent simulations for realistic clinical interactions\.External Links:2503\.22678,[Link](https://arxiv.org/abs/2503.22678)Cited by:[§2\.3](https://arxiv.org/html/2607.02983#S2.SS3.p1.1)\.
- Z\. Bao, W\. Chen, S\. Xiao, K\. Ren, J\. Wu, C\. Zhong, J\. Peng, X\. Huang, and Z\. Wei \(2023\)DISC\-medllm: bridging general large language models and real\-world medical consultation\.External Links:2308\.14346,[Link](https://arxiv.org/abs/2308.14346)Cited by:[§2\.3](https://arxiv.org/html/2607.02983#S2.SS3.p1.1)\.
- M\. Besta, N\. Blach, A\. Kubicek, R\. Gerstenberger, M\. Podstawski, L\. Gianinazzi, J\. Gajda, T\. Lehmann, H\. Niewiadomski, P\. Nyczyk, and T\. Hoefler \(2024\)Graph of thoughts: solving elaborate problems with large language models\.InProceedings of the Thirty\-Eighth AAAI Conference on Artificial Intelligence and Thirty\-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence,AAAI’24/IAAI’24/EAAI’24\.External Links:ISBN 978\-1\-57735\-887\-9,[Link](https://doi.org/10.1609/aaai.v38i16.29720),[Document](https://dx.doi.org/10.1609/aaai.v38i16.29720)Cited by:[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1)\.
- J\. Chen, Z\. Cai, K\. Ji, X\. Wang, W\. Liu, R\. Wang, J\. Hou, and B\. Wang \(2024\)HuatuoGPT\-o1, towards medical complex reasoning with llms\.External Links:2412\.18925,[Link](https://arxiv.org/abs/2412.18925)Cited by:[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1)\.
- X\. Chen, H\. Yi, M\. You, W\. Liu, L\. Wang, H\. Li, X\. Zhang, Y\. Guo, L\. Fan, G\. Chen,et al\.\(2025\)Enhancing diagnostic capability with multi\-agents conversational large language models\.NPJ digital medicine8\(1\),pp\. 159\.Cited by:[§2\.3](https://arxiv.org/html/2607.02983#S2.SS3.p1.1)\.
- Y\. Chen, Z\. Wang, X\. Xing, huimin zheng, Z\. Xu, K\. Fang, J\. Wang, S\. Li, J\. Wu, Q\. Liu, and X\. Xu \(2023\)BianQue: balancing the questioning and suggestion ability of health llms with multi\-turn health conversations polished by chatgpt\.External Links:2310\.15896,[Link](https://arxiv.org/abs/2310.15896)Cited by:[§2\.3](https://arxiv.org/html/2607.02983#S2.SS3.p1.1)\.
- DeepSeek\-AI \(2025\)DeepSeek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.External Links:2501\.12948,[Link](https://arxiv.org/abs/2501.12948)Cited by:[§1](https://arxiv.org/html/2607.02983#S1.p1.1),[§1](https://arxiv.org/html/2607.02983#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2607.02983#S4.SS1.p1.5)\.
- G\. Emmi, A\. Bettiol, E\. Gelain, I\. M\. Bajema, A\. Berti, S\. Burns, M\. C\. Cid, J\. W\. Cohen Tervaert, V\. Cottin, E\. Durante, J\. U\. Holle, A\. D\. Mahr, M\. M\. Del Pero, C\. Marvisi, J\. Mills, S\. Moiseev, F\. Moosig, C\. Mukhtyar, T\. Neumann, I\. Olivotto, C\. Salvarani, B\. Seeliger, R\. A\. Sinico, C\. Taillé, B\. Terrier, N\. Venhoff, G\. Bertsias, L\. Guillevin, D\. R\. W\. Jayne, and A\. Vaglio \(2023\)Evidence\-based guideline for the diagnosis and management of eosinophilic granulomatosis with polyangiitis\.Nature Reviews Rheumatology19\(6\),pp\. 378–393\.External Links:ISSN 1759\-4804,[Link](http://dx.doi.org/10.1038/s41584-023-00958-w),[Document](https://dx.doi.org/10.1038/s41584-023-00958-w)Cited by:[§1](https://arxiv.org/html/2607.02983#S1.p1.1)\.
- Z\. Fan, L\. Wei, J\. Tang, W\. Chen, W\. Siyuan, Z\. Wei, and F\. Huang \(2025\)AI hospital: benchmarking large language models in a multi\-agent medical interaction simulator\.InProceedings of the 31st International Conference on Computational Linguistics,O\. Rambow, L\. Wanner, M\. Apidianaki, H\. Al\-Khalifa, B\. D\. Eugenio, and S\. Schockaert \(Eds\.\),Abu Dhabi, UAE,pp\. 10183–10213\.External Links:[Link](https://aclanthology.org/2025.coling-main.680/)Cited by:[§2\.3](https://arxiv.org/html/2607.02983#S2.SS3.p1.1)\.
- M\. Y\. Guan, M\. Joglekar, E\. Wallace, S\. Jain, B\. Barak, A\. Helyar, R\. Dias, A\. Vallone, H\. Ren, J\. Wei,et al\.\(2024\)Deliberative alignment: reasoning enables safer language models\.arXiv preprint arXiv:2412\.16339\.Cited by:[§2\.2](https://arxiv.org/html/2607.02983#S2.SS2.p1.1)\.
- E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen \(2022\)LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=nZeVKeeFYf9)Cited by:[§A\.8](https://arxiv.org/html/2607.02983#A1.SS8.p4.3)\.
- J\. Hu, X\. Wu, Z\. Zhu, Xianyu, W\. Wang, D\. Zhang, and Y\. Cao \(2024\)OpenRLHF: an easy\-to\-use, scalable and high\-performance rlhf framework\.arXiv preprint arXiv:2405\.11143\.Cited by:[§4\.1](https://arxiv.org/html/2607.02983#S4.SS1.p1.5)\.
- Z\. Huang, G\. Geng, S\. Hua, Z\. Huang, H\. Zou, S\. Zhang, P\. Liu, and X\. Zhang \(2025\)O1 replication journey – part 3: inference\-time scaling for medical reasoning\.External Links:2501\.06458,[Link](https://arxiv.org/abs/2501.06458)Cited by:[§A\.8](https://arxiv.org/html/2607.02983#A1.SS8.p1.1),[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1),[§4\.2](https://arxiv.org/html/2607.02983#S4.SS2.p3.1)\.
- Hugging Face \(2025\)Open r1: a fully open reproduction of deepseek\-r1\.External Links:[Link](https://github.com/huggingface/open-r1)Cited by:[§2\.2](https://arxiv.org/html/2607.02983#S2.SS2.p1.1)\.
- A\. Jaech, A\. Kalai, A\. Lerer, A\. Richardson, A\. El\-Kishky, A\. Low, A\. Helyar, A\. Madry, A\. Beutel, A\. Carney,et al\.\(2024\)Openai o1 system card\.arXiv preprint arXiv:2412\.16720\.Cited by:[§1](https://arxiv.org/html/2607.02983#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1)\.
- S\. Jiang, Y\. Liao, Z\. Chen, Y\. Zhang, Y\. Wang, and Y\. Wang \(2025a\)MedS3: towards medical small language models with self\-evolved slow thinking\.External Links:2501\.12051,[Link](https://arxiv.org/abs/2501.12051)Cited by:[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1)\.
- Y\. Jiang, K\. C\. Black, G\. Geng, D\. Park, J\. Zou, A\. Y\. Ng, and J\. H\. Chen \(2025b\)MedAgentBench: a realistic virtual ehr environment to benchmark medical llm agents\.External Links:2501\.14654,[Link](https://arxiv.org/abs/2501.14654)Cited by:[§2\.3](https://arxiv.org/html/2607.02983#S2.SS3.p1.1)\.
- D\. Jin, E\. Pan, N\. Oufattole, W\. Weng, H\. Fang, and P\. Szolovits \(2021\)What disease does this patient have? a large\-scale open domain question answering dataset from medical exams\.Applied Sciences11\(14\),pp\. 6421\.Cited by:[§A\.4](https://arxiv.org/html/2607.02983#A1.SS4.p1.1)\.
- T\. Kwon, K\. T\. Ong, D\. Kang, S\. Moon, J\. R\. Lee, D\. Hwang, Y\. Sim, B\. Sohn, D\. Lee, and J\. Yeo \(2024\)Large language models are clinical reasoners: reasoning\-aware diagnosis framework with prompt\-generated rationales\.External Links:2312\.07399,[Link](https://arxiv.org/abs/2312.07399)Cited by:[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1)\.
- W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica \(2023\)Efficient memory management for large language model serving with pagedattention\.InProceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles,Cited by:[§A\.8](https://arxiv.org/html/2607.02983#A1.SS8.p4.3)\.
- Y\. Li, Z\. Li, K\. Zhang, R\. Dan, S\. Jiang, and Y\. Zhang \(2023\)ChatDoctor: a medical chat model fine\-tuned on a large language model meta\-ai \(llama\) using medical domain knowledge\.External Links:2303\.14070,[Link](https://arxiv.org/abs/2303.14070)Cited by:[§2\.3](https://arxiv.org/html/2607.02983#S2.SS3.p1.1)\.
- C\. Liu, D\. Li, Y\. Shu, R\. Chen, D\. Duan, T\. Fang, and B\. Dai \(2025a\)Fleming\-r1: toward expert\-level medical reasoning via reinforcement learning\.External Links:2509\.15279,[Link](https://arxiv.org/abs/2509.15279)Cited by:[§2\.2](https://arxiv.org/html/2607.02983#S2.SS2.p1.1)\.
- R\. Liu, K\. Xue, X\. Zhang, and S\. Zhang \(2025b\)Interactive evaluation for medical LLMs via task\-oriented dialogue system\.InProceedings of the 31st International Conference on Computational Linguistics,O\. Rambow, L\. Wanner, M\. Apidianaki, H\. Al\-Khalifa, B\. D\. Eugenio, and S\. Schockaert \(Eds\.\),Abu Dhabi, UAE,pp\. 4871–4896\.External Links:[Link](https://aclanthology.org/2025.coling-main.325/)Cited by:[§2\.3](https://arxiv.org/html/2607.02983#S2.SS3.p1.1)\.
- X\. Liu, H\. Liu, G\. Yang, Z\. Jiang, S\. Cui, Z\. Zhang, H\. Wang, L\. Tao, Y\. Sun, Z\. Song,et al\.\(2025c\)A generalist medical language model for disease diagnosis assistance\.Nature Medicine,pp\. 1–11\.Cited by:[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1)\.
- Z\. Liu, X\. Zhao, J\. Peng, Z\. Zhu, Q\. Chen, X\. Hu, and T\. Chen \(2025d\)Dialogue is better than monologue: instructing medical llms via strategical conversations\.External Links:2501\.17860,[Link](https://arxiv.org/abs/2501.17860)Cited by:[§2\.3](https://arxiv.org/html/2607.02983#S2.SS3.p1.1)\.
- M\. Luo, S\. Tan, J\. Wong, X\. Shi, W\. Y\. Tang, M\. Roongta, C\. Cai, J\. Luo, T\. Zhang, L\. E\. Li,et al\.\(2025\)Deepscaler: surpassing o1\-preview with a 1\.5 b model by scaling rl\.Notion Blog\.Cited by:[§2\.2](https://arxiv.org/html/2607.02983#S2.SS2.p1.1)\.
- D\. McDuff, M\. Schaekermann, T\. Tu, A\. Palepu, A\. Wang, J\. Garrison, K\. Singhal, Y\. Sharma, S\. Azizi, K\. Kulkarni,et al\.\(2025\)Towards accurate differential diagnosis with large language models\.Nature,pp\. 1–7\.Cited by:[§4\.2](https://arxiv.org/html/2607.02983#S4.SS2.p2.1)\.
- Y\. Mroueh \(2025\)Reinforcement learning with verifiable rewards: grpo’s effective loss, dynamics, and success amplification\.arXiv preprint arXiv:2503\.06639\.Cited by:[§1](https://arxiv.org/html/2607.02983#S1.p3.1),[§2\.2](https://arxiv.org/html/2607.02983#S2.SS2.p1.1)\.
- H\. Nori, Y\. T\. Lee, S\. Zhang, D\. Carignan, R\. Edgar, N\. Fusi, N\. King, J\. Larson, Y\. Li, W\. Liu, R\. Luo, S\. M\. McKinney, R\. O\. Ness, H\. Poon, T\. Qin, N\. Usuyama, C\. White, and E\. Horvitz \(2023\)Can generalist foundation models outcompete special\-purpose tuning? case study in medicine\.External Links:2311\.16452,[Link](https://arxiv.org/abs/2311.16452)Cited by:[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1)\.
- H\. Nori, N\. Usuyama, N\. King, S\. M\. McKinney, X\. Fernandes, S\. Zhang, and E\. Horvitz \(2024\)From medprompt to o1: exploration of run\-time strategies for medical challenge problems and beyond\.External Links:2411\.03590,[Link](https://arxiv.org/abs/2411.03590)Cited by:[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1)\.
- P\. Qiu, C\. Wu, J\. Liu, Q\. Zheng, Y\. Liao, H\. Wang, Y\. Yue, Q\. Fan, S\. Zhen, J\. Wang, J\. Gu, Y\. Wang, Y\. Zhang, and W\. Xie \(2025a\)Evolving diagnostic agents in a virtual clinical environment\.External Links:2510\.24654,[Link](https://arxiv.org/abs/2510.24654)Cited by:[§2\.3](https://arxiv.org/html/2607.02983#S2.SS3.p1.1),[§4\.2](https://arxiv.org/html/2607.02983#S4.SS2.p3.1)\.
- Z\. Qiu, Z\. Zhang, Y\. Hu, H\. Li, and J\. Liu \(2025b\)Open\-medical\-r1: how to choose data for rlvr training at medicine domain\.External Links:2504\.13950,[Link](https://arxiv.org/abs/2504.13950)Cited by:[§1](https://arxiv.org/html/2607.02983#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.02983#S2.SS2.p1.1)\.
- QwenTeam \(2025\)QwQ\-32b: embracing the power of reinforcement learning\.External Links:[Link](https://qwenlm.github.io/blog/qwq-32b/)Cited by:[§4\.2](https://arxiv.org/html/2607.02983#S4.SS2.p3.1)\.
- S\. Sandmann, S\. Hegselmann, M\. Fujarski, L\. Bickmann, B\. Wild, R\. Eils, and J\. Varghese \(2025\)Benchmark evaluation of deepseek large language models in clinical decision\-making\.Nature Medicine,pp\. 1–1\.Cited by:[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1)\.
- T\. Savage, A\. Nayak, R\. Gallo, E\. Rangan, and J\. H\. Chen \(2024\)Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine\.NPJ Digital Medicine7\(1\),pp\. 20\.Cited by:[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1)\.
- K\. Singhal, S\. Azizi, T\. Tu, S\. S\. Mahdavi, J\. Wei, H\. W\. Chung, N\. Scales, A\. Tanwani, H\. Cole\-Lewis, S\. Pfohl,et al\.\(2023\)Large language models encode clinical knowledge\.Nature620\(7972\),pp\. 172–180\.Cited by:[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1)\.
- Z\. Sun, C\. Luo, Z\. Liu, and Z\. Huang \(2024\)Conversational disease diagnosis via external planner\-controlled large language models\.arXiv preprint arXiv:2404\.04292\.Cited by:[§2\.3](https://arxiv.org/html/2607.02983#S2.SS3.p1.1)\.
- M\. F\. B\. Tarek and R\. Beheshti \(2025\)Reward hacking mitigation using verifiable composite rewards\.External Links:2509\.15557,[Link](https://arxiv.org/abs/2509.15557)Cited by:[§2\.2](https://arxiv.org/html/2607.02983#S2.SS2.p1.1),[§3\.1\.3](https://arxiv.org/html/2607.02983#S3.SS1.SSS3.p1.1)\.
- M\. Team, C\. Dou, C\. Liu, F\. Yang, F\. Li, J\. Jia, M\. Chen, Q\. Ju, S\. Wang, S\. Dang, T\. Li, X\. Zeng, and e\. al\. Yijie Zhou \(2025\)Baichuan\-m2: scaling medical capability with large verifier system\.External Links:2509\.02208,[Link](https://arxiv.org/abs/2509.02208)Cited by:[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1)\.
- A\. Toma, P\. R\. Lawler, J\. Ba, R\. G\. Krishnan, B\. B\. Rubin, and B\. Wang \(2023\)Clinical camel: an open expert\-level medical language model with dialogue\-based knowledge encoding\.External Links:2305\.12031,[Link](https://arxiv.org/abs/2305.12031)Cited by:[§2\.3](https://arxiv.org/html/2607.02983#S2.SS3.p1.1)\.
- M\. Tordjman, Z\. Liu, M\. Yuce, V\. Fauveau, Y\. Mei, J\. Hadjadj, I\. Bolger, H\. Almansour, C\. Horst, A\. S\. Parihar,et al\.\(2025\)Comparative benchmarking of the deepseek large language model on medical tasks and clinical reasoning\.Nature Medicine,pp\. 1–1\.Cited by:[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1)\.
- T\. Tu, A\. Palepu, M\. Schaekermann, K\. Saab, J\. Freyberg, R\. Tanno, A\. Wang, B\. Li, M\. Amin, N\. Tomasev, S\. Azizi, K\. Singhal, Y\. Cheng, L\. Hou, A\. Webson, K\. Kulkarni, S\. S\. Mahdavi, C\. Semturs, J\. Gottweis, J\. Barral, K\. Chou, G\. S\. Corrado, Y\. Matias, A\. Karthikesalingam, and V\. Natarajan \(2024\)Towards conversational diagnostic ai\.External Links:2401\.05654,[Link](https://arxiv.org/abs/2401.05654)Cited by:[§2\.3](https://arxiv.org/html/2607.02983#S2.SS3.p1.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. Chi, Q\. Le, and D\. Zhou \(2023\)Chain\-of\-thought prompting elicits reasoning in large language models\.External Links:2201\.11903,[Link](https://arxiv.org/abs/2201.11903)Cited by:[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. Qiu \(2025\)Qwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§4\.2](https://arxiv.org/html/2607.02983#S4.SS2.p3.1)\.
- A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. Qiu \(2024\)Qwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.Cited by:[§A\.8](https://arxiv.org/html/2607.02983#A1.SS8.p4.3),[§4\.1](https://arxiv.org/html/2607.02983#S4.SS1.p1.5)\.
- S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. L\. Griffiths, Y\. Cao, and K\. Narasimhan \(2023\)Tree of thoughts: deliberate problem solving with large language models\.InProceedings of the 37th International Conference on Neural Information Processing Systems,NIPS ’23,Red Hook, NY, USA\.Cited by:[§2\.1](https://arxiv.org/html/2607.02983#S2.SS1.p1.1)\.
- S\. Zhang, Q\. Liu, G\. Qin, T\. Naumann, and H\. Poon \(2025\)Med\-rlvr: emerging medical reasoning from a 3b base model via reinforcement learning\.arXiv preprint arXiv:2502\.19655\.Cited by:[§1](https://arxiv.org/html/2607.02983#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.02983#S2.SS2.p1.1)\.
- Y\. Zheng, R\. Zhang, J\. Zhang, Y\. Ye, Z\. Luo, Z\. Feng, and Y\. Ma \(2024\)LlamaFactory: unified efficient fine\-tuning of 100\+ language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 3: System Demonstrations\),Bangkok, Thailand\.External Links:[Link](http://arxiv.org/abs/2403.13372)Cited by:[§A\.8](https://arxiv.org/html/2607.02983#A1.SS8.p4.3)\.
- J\. Zhu and J\. Wu \(2025\)Ask patients with patience: enabling llms for human\-centric medical dialogue with grounded reasoning\.External Links:2502\.07143,[Link](https://arxiv.org/abs/2502.07143)Cited by:[§2\.3](https://arxiv.org/html/2607.02983#S2.SS3.p1.1)\.
- W\. Zhu, X\. Dong, X\. Li, P\. Qiu, X\. Chen, A\. Razi, A\. Sotiras, Y\. Su, and Y\. Wang \(2025\)Toward effective reinforcement learning fine\-tuning for medical vqa in vision\-language models\.External Links:2505\.13973,[Link](https://arxiv.org/abs/2505.13973)Cited by:[§2\.2](https://arxiv.org/html/2607.02983#S2.SS2.p1.1)\.

## Appendix AAppendix

### A\.1Proof

Theorem[3\.1](https://arxiv.org/html/2607.02983#S3.Thmtheorem1)\(Clinical Priority\)\.The diagnostic reward confers higher values when the correct diagnosis appears earlier in the list\. Consider two lists of equal length \(\|𝒟1\|=\|𝒟2\|\|\\mathcal\{D\}^\{1\}\|=\|\\mathcal\{D\}^\{2\}\|\) that both contain the correct diagnosisYY\. Let the rank ofYYin𝒟1\\mathcal\{D\}^\{1\}beiiand in𝒟2\\mathcal\{D\}^\{2\}bejj, such thatDi1=YD\_\{i\}^\{1\}=YandDj2=YD\_\{j\}^\{2\}=Ywith1≤i<j≤\|𝒟1\|=\|𝒟2\|1\\leq i<j\\leq\|\\mathcal\{D\}^\{1\}\|=\|\\mathcal\{D\}^\{2\}\|\. In this case,Rd​\(𝒟1,Y\)\>Rd​\(𝒟2,Y\)\>0R\_\{d\}\(\\mathcal\{D\}^\{1\},Y\)\>R\_\{d\}\(\\mathcal\{D\}^\{2\},Y\)\>0\.

###### Proof\.

According to the definition in Equation[2](https://arxiv.org/html/2607.02983#S3.E2), we have,

Rd​\(𝒟1,Y\)=e−i/τ∑k=1\|𝒟1\|e−k/τ=e−i/τ∑k=1\|𝒟2\|e−k/τ\>e−j/τ∑k=1\|𝒟2\|e−k/τ=Rd​\(𝒟2,Y\)R\_\{d\}\(\\mathcal\{D\}^\{1\},Y\)=\\frac\{e^\{\-i/\\tau\}\}\{\\sum\_\{k=1\}^\{\|\\mathcal\{D\}^\{1\}\|\}e^\{\-k/\\tau\}\}=\\frac\{e^\{\-i/\\tau\}\}\{\\sum\_\{k=1\}^\{\|\\mathcal\{D\}^\{2\}\|\}e^\{\-k/\\tau\}\}\>\\frac\{e^\{\-j/\\tau\}\}\{\\sum\_\{k=1\}^\{\|\\mathcal\{D\}^\{2\}\|\}e^\{\-k/\\tau\}\}=R\_\{d\}\(\\mathcal\{D\}^\{2\},Y\)∎

Theorem[3\.2](https://arxiv.org/html/2607.02983#S3.Thmtheorem2)\(Diagnostic Focus\)\.The diagnostic reward assigns higher values when the list contains a correct diagnosis and is shorter\. Consider two lists𝒟1\\mathcal\{D\}^\{1\}and𝒟2\\mathcal\{D\}^\{2\}of different lengths \(\|𝒟1\|<\|𝒟2\|\|\\mathcal\{D\}^\{1\}\|<\|\\mathcal\{D\}^\{2\}\|\) that both contain the ground truth diagnosisYY\. Let the position ofYYbeiiin both lists, such thatDi1=Di2=YD\_\{i\}^\{1\}=D\_\{i\}^\{2\}=Ywith1≤i≤\|𝒟1\|<\|𝒟2\|1\\leq i\\leq\|\\mathcal\{D\}^\{1\}\|<\|\\mathcal\{D\}^\{2\}\|\. In this case,Rd​\(𝒟1,Y\)\>Rd​\(𝒟2,Y\)\>0R\_\{d\}\(\\mathcal\{D\}^\{1\},Y\)\>R\_\{d\}\(\\mathcal\{D\}^\{2\},Y\)\>0\.

###### Proof\.

According to the definition in Equation[2](https://arxiv.org/html/2607.02983#S3.E2), we have,

Rd​\(𝒟1,Y\)=e−i/τ∑k=1\|𝒟1\|e−k/τ\>e−i/τ∑k=1\|𝒟1\|e−k/τ\+∑k=\|𝒟1\+1\|\|𝒟2\|e−k/τ=e−i/τ∑k=1\|𝒟2\|e−k/τ=Rd​\(𝒟2,Y\)R\_\{d\}\(\\mathcal\{D\}^\{1\},Y\)=\\frac\{e^\{\-i/\\tau\}\}\{\\sum\_\{k=1\}^\{\|\\mathcal\{D\}^\{1\}\|\}e^\{\-k/\\tau\}\}\>\\frac\{e^\{\-i/\\tau\}\}\{\\sum\_\{k=1\}^\{\|\\mathcal\{D\}^\{1\}\|\}e^\{\-k/\\tau\}\+\\sum\_\{k=\|\\mathcal\{D\}^\{1\}\+1\|\}^\{\|\\mathcal\{D\}^\{2\}\|\}e^\{\-k/\\tau\}\}=\\frac\{e^\{\-i/\\tau\}\}\{\\sum\_\{k=1\}^\{\|\\mathcal\{D\}^\{2\}\|\}e^\{\-k/\\tau\}\}=R\_\{d\}\(\\mathcal\{D\}^\{2\},Y\)∎

Finding[3\.3](https://arxiv.org/html/2607.02983#S3.Thmtheorem3)\(Exploratory Behavior\)\.A lowerτ\\tauin the diagnostic reward encourages the correct diagnosis to the top of the list during the exploratory phase\. Consider a fixed diagnosis list𝒟\\mathcal\{D\}that contains the ground truth diagnosisYY, where the position ofYYis 1 \(i\.e\., the top\-1 diagnosis\)\. For two hyperparameters satisfyingτ1<τ2\\tau\_\{1\}<\\tau\_\{2\}, we haveRdτ1≥Rdτ2R\_\{d\}^\{\\tau\_\{1\}\}\\geq R\_\{d\}^\{\\tau\_\{2\}\}\. The equality holds if and only if\|𝒟\|=1\|\\mathcal\{D\}\|=1\.

###### Proof\.

Letτ2=τ1\+Δ​τ\\tau\_\{2\}=\\tau\_\{1\}\+\\Delta\\tau\. After simplification, we have,

Rdτ1−Rdτ2=∑k=2\|𝒟\|e−\(τ1\+k​τ1\)/τ1​τ2​\(e−Δ​τ/τ1​τ2−e−k​Δ​τ/τ1​τ2\)∑j=1\|𝒟\|e−j/τ1​∑j=1\|𝒟\|e−j/τ2R\_\{d\}^\{\\tau\_\{1\}\}\-R\_\{d\}^\{\\tau\_\{2\}\}=\\frac\{\\sum\_\{k=2\}^\{\|\\mathcal\{D\}\|\}e^\{\-\(\\tau\_\{1\}\+k\\tau\_\{1\}\)/\\tau\_\{1\}\\tau\_\{2\}\}\(e^\{\-\\Delta\\tau/\\tau\_\{1\}\\tau\_\{2\}\}\-e^\{\-k\\Delta\\tau/\\tau\_\{1\}\\tau\_\{2\}\}\)\}\{\\sum\_\{j=1\}^\{\|\\mathcal\{D\}\|\}e^\{\-j/\\tau\_\{1\}\}\\sum\_\{j=1\}^\{\|\\mathcal\{D\}\|\}e^\{\-j/\\tau\_\{2\}\}\}\.

When\|𝒟\|\>1\|\\mathcal\{D\}\|\>1, we haveRdτ1−Rdτ2\>0R\_\{d\}^\{\\tau\_\{1\}\}\-R\_\{d\}^\{\\tau\_\{2\}\}\>0, and when\|𝒟\|=1\|\\mathcal\{D\}\|=1,Rdτ1=Rdτ2=1R\_\{d\}^\{\\tau\_\{1\}\}=R\_\{d\}^\{\\tau\_\{2\}\}=1\. ∎

Finding[3\.4](https://arxiv.org/html/2607.02983#S3.Thmtheorem4)\(Exploitative Behavior\)\.A higherτ\\tauencourages the generation of shorter diagnosis lists\. Consider two lists of differing lengths, where\|𝒟1\|<\|𝒟2\|\|\\mathcal\{D\}^\{1\}\|<\|\\mathcal\{D\}^\{2\}\|, and let the correct diagnosisYYappear at positioniiin both lists\. Define the reward increment with respect toτ\\tauasΔ​Rdτ=Rdτ​\(𝒟1,Y\)−Rdτ​\(𝒟2,Y\)\\Delta R\_\{d\}^\{\\tau\}=R\_\{d\}^\{\\tau\}\(\\mathcal\{D\}^\{1\},Y\)\-R\_\{d\}^\{\\tau\}\(\\mathcal\{D\}^\{2\},Y\)\. For hyperparameters satisfyingτ1\>τ2\\tau\_\{1\}\>\\tau\_\{2\}, we haveΔ​Rdτ1\>Δ​Rdτ2\>0\\Delta R\_\{d\}^\{\\tau\_\{1\}\}\>\\Delta R\_\{d\}^\{\\tau\_\{2\}\}\>0\.

###### Proof\.

We haveΔ​Rdτ=Rd​\(𝒟1,Y\)−Rd​\(𝒟2,Y\)=e−i/τS1,a​S1,b​Sa\+1,b\>0\\Delta R\_\{d\}^\{\\tau\}=R\_\{d\}\(\\mathcal\{D\}^\{1\},Y\)\-R\_\{d\}\(\\mathcal\{D\}^\{2\},Y\)=\\frac\{e^\{\-i/\\tau\}\}\{S\_\{1,a\}S\_\{1,b\}\}S\_\{a\+1,b\}\>0\(proved in Theorem[3\.2](https://arxiv.org/html/2607.02983#S3.Thmtheorem2)\), whereSa,b=∑k=abe−k/τS\_\{a,b\}=\\sum\_\{k=a\}^\{b\}\{e^\{\-k/\\tau\}\},a=\|𝒟1\|a=\|\\mathcal\{D\}^\{1\}\|andb=\|𝒟2\|b=\|\\mathcal\{D\}^\{2\}\|\.

Its log\-derivation is\(log⁡Δ​Rdτ\)′=iτ2\+1τ2​\(Ta\+1,b−T1,a−T1,b\)\(\\log\\Delta R\_\{d\}^\{\\tau\}\)^\{\\prime\}=\\frac\{i\}\{\\tau^\{2\}\}\+\\frac\{1\}\{\\tau^\{2\}\}\\left\(T\_\{a\+1,b\}\-T\_\{1,a\}\-T\_\{1,b\}\\right\), whereTa,b=∑k=abk​e−k/τSa,bT\_\{a,b\}=\\frac\{\\sum\_\{k=a\}^\{b\}ke^\{\-k/\\tau\}\}\{S\_\{a,b\}\}\.

With several simplifications, we haveF=\(1−T1,a\)​S1,b\+\(Ta\+1,b−T1,a\)​S1,aF=\(1\-T\_\{1,a\}\)S\_\{1,b\}\+\(T\_\{a\+1,b\}\-T\_\{1,a\}\)S\_\{1,a\}, which has the same sign with\(log⁡Δ​Rdτ\)′\(\\log\\Delta R\_\{d\}^\{\\tau\}\)^\{\\prime\}\.

It is clear thatS1,aS\_\{1,a\}is a geometric sequence with the initial iteme−1/τe^\{\-1/\\tau\}and a ratioq=e−1/τq=e^\{\-1/\\tau\}\. By using the summation formula, we haveS1,a=q​\(1−qa\)1−qS\_\{1,a\}=\\frac\{q\(1\-q^\{a\}\)\}\{1\-q\}\.

Then we have

F2\\displaystyle F\_\{2\}=Fq/\(1−q\)\\displaystyle=\\frac\{F\}\{q/\(1\-q\)\}=\(1−T1,a\)​\(1−qb\)\+\(Ta\+1,b−T1,a\)​\(1−qa\)\\displaystyle=\(1\-T\_\{1,a\}\)\(1\-q^\{b\}\)\+\(T\_\{a\+1,b\}\-T\_\{1,a\}\)\(1\-q^\{a\}\)=\(1−T1,a\+Ta\+1,b−T1,a\)−qb​\(1−T1,a\)−qa​\(Ta\+1,b−T1,a\)\\displaystyle=\(1\-T\_\{1,a\}\+T\_\{a\+1,b\}\-T\_\{1,a\}\)\-q^\{b\}\(1\-T\_\{1,a\}\)\-q^\{a\}\(T\_\{a\+1,b\}\-T\_\{1,a\}\)\>\(1−T1,a\+Ta\+1,b−T1,a\)−qa​\(1−T1,a\)−qa​\(Ta\+1,b−T1,a\)\\displaystyle\>\(1\-T\_\{1,a\}\+T\_\{a\+1,b\}\-T\_\{1,a\}\)\-q^\{a\}\(1\-T\_\{1,a\}\)\-q^\{a\}\(T\_\{a\+1,b\}\-T\_\{1,a\}\)=\(1\+Ta\+1,b−2​T1,a\)​\(1−qa\)\\displaystyle=\(1\+T\_\{a\+1,b\}\-2T\_\{1,a\}\)\(1\-q^\{a\}\)
Ta,bT\_\{a,b\}can be seen as a weighted sum over\{k\|a≤k≤b\}\\\{k\|a\\leq k\\leq b\\\}with monotonically decreasing weights\{e−k/τ\|a≤k≤b\}\\\{e^\{\-k/\\tau\}\|a\\leq k\\leq b\\\}\. Therefore,Ta,bT\_\{a,b\}should be larger than the lower boundaawhile less than its arithmetic meana\+b2\\frac\{a\+b\}\{2\}\.

Hence we haveTa\+1,b\>a\+1=2​a\+12\>2​T1,aT\_\{a\+1,b\}\>a\+1=2\\frac\{a\+1\}\{2\}\>2T\_\{1,a\}\. By using all things ahead, we have,

\(log⁡Δ​Rdτ\)′\\displaystyle\(\\log\\Delta R\_\{d\}^\{\\tau\}\)^\{\\prime\}∝F∝F2\\displaystyle\\propto F\\propto F\_\{2\}\>\(1\+Ta\+1,b−2​T1,a\)​\(1−qa\)\\displaystyle\>\(1\+T\_\{a\+1,b\}\-2T\_\{1,a\}\)\(1\-q^\{a\}\)\>1−qa\>0\\displaystyle\>1\-q^\{a\}\>0
Therefore,Δ​Rdτ\\Delta R\_\{d\}^\{\\tau\}is strictly monotonically increasing withτ\\tau, and thus we haveΔ​Rdτ1\>Δ​Rdτ2\>0\\Delta R\_\{d\}^\{\\tau\_\{1\}\}\>\\Delta R\_\{d\}^\{\\tau\_\{2\}\}\>0whenτ1\>τ2\\tau\_\{1\}\>\\tau\_\{2\}\.

∎

![Refer to caption](https://arxiv.org/html/2607.02983v1/x3.png)Figure 5:Visualization of the proposed rank\-sensitive diagnosis reward\.Left: the rewards of different positions under various diagnosis list lengths with a fixedτ=1\.0\\tau=1\.0;Middle: the rewards under a fixed length\|𝒟\|=5\|\\mathcal\{D\}\|=5with differentτ\\tau;Right: the reward gains when the length shortens as\|𝒟\|=5→4\|\\mathcal\{D\}\|=5\\to 4with differentτ\\tau\.
### A\.2Visualization of Rank\-sensitive Diagnostic Reward

Fig\.[5](https://arxiv.org/html/2607.02983#A1.F5)visualizes three representative charts\. The left panel instantiates the rank\-sensitive diagnosis rewards at different hit positions across varying list lengths under a fixedτ\\tau, highlighting the first two properties \(Theorems[3\.1](https://arxiv.org/html/2607.02983#S3.Thmtheorem1)and[3\.2](https://arxiv.org/html/2607.02983#S3.Thmtheorem2)\)\. The middle panel illustrates Finding[3\.3](https://arxiv.org/html/2607.02983#S3.Thmtheorem3)by comparing different choices ofτ\\tauwith a fixed list length\. The right panel shows the reward gains obtained when the list length decreases from 5 to 4 across differentτ\\tau, consistent with Finding[3\.4](https://arxiv.org/html/2607.02983#S3.Thmtheorem4)\.

Table 6:Performance of models under different settings: a direct diagnosis fashion and the proposed evidence\-seeking paradigm\.Boldfigures suggest the best performance, and theunderlinedare the second best\. For cells that contain two numbers, the first represents the performance, and the subscript indicates the change compared with the direct diagnosis setting \(positive values ingreen, negative inred\)\. For clarity and consistency, all subscripts are rounded to two significant digits\.Direct DxAccEvidence\-seeking DxAccEvidence\-seeking DiffAccModelENCNMeanENCNMeanENCNMeanQwen2\.5\-32B33\.931\.732\.324\.59\.434\.22\.531\.40\.942\.78\.851\.920\.49\.317\.Qwen2\.5\-72B43\.337\.339\.043\.90\.645\.48\.045\.06\.054\.211\.65\.628\.62\.423\.QwQ\-32B37\.3\\ul43\.741\.9\\ul42\.75\.4\\ul50\.46\.7\\ul48\.26\.3\\ul57\.620\.\\ul66\.122\.\\ul63\.622\.Qwen3\-32B36\.042\.740\.829\.76\.343\.10\.439\.31\.544\.38\.356\.314\.52\.812\.Huatuo\-7B22\.424\.023\.619\.72\.721\.12\.920\.72\.931\.59\.140\.116\.37\.614\.M2\-32B37\.644\.3\\ul42\.437\.90\.349\.24\.946\.03\.652\.715\.66\.022\.62\.220\.Ours\-SFT\-32B\\ul40\.943\.642\.8\\ul42\.71\.851\.37\.748\.96\.154\.914\.67\.023\.\\ul63\.621\.Ours\-RL\-7B29\.134\.432\.936\.47\.339\.04\.638\.25\.362\.033\.65\.831\.64\.832\.
### A\.3Idealized Experiment: One\-turn Diagnosis with More Evidence

To better mitigate the information discrepancy, we conduct an idealized one\-turn diagnosis experiment in which models are provided with as much clinical information as possible\. This setting is intentionally “ideal," as real\-world clinical diagnosis does not permit access to future follow\-up results\. The outcomes are summarized in Table[6](https://arxiv.org/html/2607.02983#A1.T6)\.

Overall, evidence\-seeking remains beneficial for most models\. However, Qwen2\.5\-32B and Qwen3\-32B achieve higher accuracy when directly prompted to provide diagnoses for the English cases\. A plausible explanation is that these English cases contain sufficient examination findings \(whereas some Chinese cases may lack such details\), which are better suited to these baseline models, not tailored to an evidence\-seeking paradigm\. Huatuo\-7B, which shows consistently degraded performance, may suffer from strong adherence to its training patterns, i\.e\., the traditional one\-turn diagnosis\. This issue is also observed in the main experiments, where it tends not to follow the instructed output format but defaults to its own pattern\.

In contrast, the differential accuracy provides a more encouraging signal\. It can be viewed as an approximate upper bound for the evidence\-seeking paradigm, suggesting substantial room for improvement: not only by enriching the available information but also by enhancing models’ ability to integrate and leverage appended clinical evidence effectively\.

### A\.4Performance on General Medical Tasks: MedQA

To evaluate whether task\-specific training influences general performance on medical tasks, we report the results of our models alongside their corresponding base models on the MedQA dataset\(Jinet al\.,[2021](https://arxiv.org/html/2607.02983#bib.bib51)\), including both the English and Simplified Chinese test subsets \(Table[7](https://arxiv.org/html/2607.02983#A1.T7)\)\.

For the RL\-trained model, performance differences are minimal, suggesting that its original medical capabilities are largely retained\. In contrast, the SFT\-trained model exhibits larger differences\. Nevertheless, likely due to the narrow focus on a single diagnostic task and the limited number of cases, we do not observe a general improvement in zero\-shot medical QA performance\.

Table 7:Performance of models on MedQA zero\-shot QA task\. For cells that contain two numbers, the first represents the performance, and the subscript indicates the change compared with the base model \(positive values ingreen, negative inred\)\. For clarity and consistency, all subscripts are rounded to two significant digits\.
### A\.5Performance ofOurs\-SFT\-7B

To further investigate the performance differences between SFT and RL models, we trained an SFT model of the same size \(7B\) as the RL model\. Table[8](https://arxiv.org/html/2607.02983#A1.T8)reports the performance of Huatuo\-7B \(as a baseline\), Ours\-SFT\-7B, and Ours\-RL\-7B on both English \(EN\) and Chinese \(CN\) test sets, in terms of differential diagnosis accuracy \(DiffAcc\) and final diagnosis accuracy \(DxAcc\)\. From the results, several observations can be made:

Performance gains from task\-specific training\. Both Ours\-SFT\-7B and Ours\-RL\-7B substantially outperform the Huatuo\-7B baseline trained with general medical data across all metrics\. Moreover, RL training yields the most significant improvements\. Compared with SFT, RL training produces notable additional gains in both DiffAcc and DxAcc\. Specifically, Ours\-RL\-7B improves the mean DxAcc from 30\.1 \(SFT\) to 38\.2, indicating that reinforcement learning better aligns the model with the rank\-sensitive reward and evidence\-seeking behavior, resulting in more accurate final diagnoses\.

Table 8:Performance comparison of Huatuo\-7B, Ours\-SFT\-7B and Ours\-RL\-7B\.
### A\.6Discussion on Efficiency

In the main text, we report that the training process takes approximately 40 hours on 8 H100 GPUs\. Time consumption arises from two main sources: the basic RL training \(rollout generation, gradient computation, etc\.\) and the computationally intensive model\-based verification process \(diagnosis and examination assessment\)\. We provide further details below\.

Inefficient GPU usage with LLM as Verifier\.Using collocation and sleep strategies, OpenRLHF efficiently allocates GPUs across models with different roles\. However, we could not identify a feasible interface to integrate an additional model for judgment that directly returns text\. Consequently, we adopted a less efficient approach: we dedicated 4 GPUs exclusively to the reasoning verifier and relied on HTML to transfer information between models\.

LLM verifiers are computationally expensive\.Beyond the framework overhead, the verifiers themselves incur significant cost\. For each generation, the verifier is used twice: once for diagnostic evaluation to promote accurate diagnoses, and once for examination assessment to ensure reasonable test requests, effectively doubling the computational load\. In a trial experiment without the examination judge, training time dropped to roughly 28 hours\. Using a smaller verifier, such as Qwen2\.5\-7B, nearly halved the judging cost, but led to severe counting errors and incorrect judgments\. Therefore, we ultimately hosted a Qwen2\.5\-32B model on 4 GPUs for the judging role, despite the inefficiency\.

Table 9:Inference efficiency of models\. Hosted on 4 H100 GPUs with vLLM engine\.For inference efficiency, we report the average inference speed \(items per second, tested on 4 H100 GPUs with vLLM engines\) and the average number of generated tokens per model in Table[9](https://arxiv.org/html/2607.02983#A1.T9)\.

From the results, several observations can be made:

Reasoning models typically generate more tokens than chat models\.For instance, Ours\-SFT\-32B produces 2244\.7 tokens in the initial turn compared with 414\.1 tokens for Qwen2\.5\-32B\. This increase in token generation naturally slows down the inference speed, as seen in Ours\-SFT\-32B’s lower speed \(1\.27 it/s\) relative to chat models like Ours\-SFT\-32B \(6\.02 it/s\)\.

Follow\-up turns often generate fewer tokens in some models, but our models show the opposite tendency\.For example, Qwen3\-32B generates 1455\.5 tokens in the initial turn at 4\.21 it/s and only 824\.1 tokens in the follow\-up turn at 5\.38 it/s\. In contrast, Ours\-SFT\-32B produces more tokens in the follow\-up \(2462\.4\) than in the initial turn \(2244\.7\), which slightly slows down its speed \(2\.02 it/s\)\. This may suggest the verbosity of our models in the follow\-up stage\.

Taking an additional turn generally increases total computational consumption\.The follow\-up turn incurs the least overhead with M2\-32B, achieving roughly 5x speed and generating 32\.8% more tokens\. Nevertheless, this additional cost remains non\-negligible\.

### A\.7Prompts in Use

Here is the prompt used in RL training\.

\{tcblisting\}

colback=gray\!10, colframe=black, title=RL Training Template, fonttitle=, coltitle=black, breakable, listing only, enhanced, sharp corners

system: You are Qwen, created by Alibaba Cloud\. You are a helpful assistant\. A conversation between User and Assistant\. The user asks a question, and the Assistant solves it\. The assistant first drafts the reasoning process \(inner monologue\) until it has derived the final answer with full confidence\. It then provides a self\-contained summary of the thoughts, i\.e\., keeping succinct but containing all the critical steps needed to reach the conclusion\. It should use Markdown and Latex to format the response\. Write both the thoughts and summary in the same language as the task posed by the user\.\\n\\nThe thinking process must follow the template below \(You should \*\*include and only include one\*\* pair of <think\></think\> and <answer\></answer\> tags in your response\):\\n<think\>\\nThe thoughts or/and draft, like working through an exercise on scratch paper\. Be as casual and as long as necessary until it is confident to generate a correct answer\.\\n</think\>\\n\\n<answer\>\\nHere, provide a concise summary that reflects the reasoning process and presents a clear final answer to the user\.\\n</answer\>\\n

user: I need you to infer possible differential diagnoses and related additional tests based on the provided case information\. Specifically, you need to analyze the given case information carefully\. Then, based on the evidence points within it, gradually deduce all possible related differential diagnoses \(these diagnoses must be relatively specific, avoiding simple descriptions such as "benign/malignant lesion"\)\. Then, re\-examine the case information to exclude those differential options with extremely low probability \(actively reflect: does the existing information contain anything that clearly conflicts with this disease?\)\. At the same time, you may also add new differential diagnoses \(actively reflect: have I missed any possibilities?\)\. You can engage in such reflection even during the initial inference\. After repeating this process for multiple rounds, when you are sufficiently confident that the current set of differential diagnoses has a high probability of including the final diagnosis while having an extremely low likelihood of including unrelated diagnoses, you may proceed to the final summary stage\. When summarizing the final differential diagnoses, rank those with a higher probability first\. Note that these differential diagnoses should not exceed ten\. Then, based on these differential diagnoses, provide the additional information needed to rule out some of the possible diagnoses \(such as a specific immunohistochemical antibody, a specialized histochemical stain, or a particular molecular test\)\. While thinking, you may review previous content at any time for reflection and promptly revise your conclusions as needed\. After thinking, you need to organize your response: briefly summarize your thought process, then summarize your output in the specified format\. Format requirements are as follows: Differential Diagnoses:\\DiffListDiagnosis 1, Diagnosis 2, … Further Examination Items:\\ExamListItem 1, Item 2, … The following is the case information: <Case Information\>

Here is the full Reasoning Verifier prompt for the proposed diagnosis\.

\{tcblisting\}

colback=gray\!10, colframe=black, title=Diagnosis Judging Prompt, fonttitle=, coltitle=black, breakable, listing only, enhanced, sharp corners

I need you to act as a professional pathologist\. After understanding what diseases the given candidate diagnoses and the actual diagnosis each represent \(note: you only need to focus on the primary diagnostic conclusion in the actual diagnosis\), analyze one by one whether each candidate diagnosis is the same as the actual diagnosis\. To help you make a better judgment, I will also provide you with the differential diagnoses related to the actual diagnosis \(the actual diagnosis is included among them\)\. When a candidate diagnosis aligns with most of the differential diagnoses, you cannot consider that candidate diagnosis to be consistent with the actual diagnosis\. You also need to record the corresponding position\. If the candidate diagnoses contain a diagnosis that is clearly contradictory to the actual diagnosis \(e\.g\., the candidates include testicular cancer but the true diagnosis is ovarian cancer\), or if a candidate largely aligns with most of the differential diagnoses, or if it refers to a disease that does not actually exist \(e\.g\., "Aba Aba lymphoma"\), the position is recorded as \-1\. Next, if there is no identical candidate diagnosis, but the candidate is related to the actual diagnosis in some way \(easily confused / anatomically close / similar manifestation / easy to associate\), then the position is recorded as 0\. If there is an identical diagnosis among the candidates, calculate the position of the matching diagnosis among all candidate diagnoses \(the position of the first one is 1, followed by 2, 3, …\)\. You also need to record the total number of all candidate diagnoses\. Output format requirements: After analyzing all candidate diagnoses, separate your analysis from the final result with —\. Then, if there is an identical diagnosis, output exactly one line:T​r​u​e\+\\posposition / total; if there is none, outputF​a​l​s​e\+\\pos0 / total; if there is a clearly contradictory diagnosis, outputF​a​l​s​e\+\\pos\-1 / total; if the input is None, outputF​a​l​s​e\+\\pos0 / 0\. Ground truth diagnosis: … Reference differential diagnoses: … Candidate diagnoses: …

Here is the full Reasoning Verifier prompt for proposed examinations\.

\{tcblisting\}

colback=gray\!10, colframe=black, title=Examination Judging Prompt, fonttitle=, coltitle=black, breakable, listing only, enhanced, sharp cornersI need you to act as a professional pathologist to help me determine whether the provided examination items can assist in differentiating among the given differential diagnoses\. At the same time, I need you to assess whether the items are valid \(i\.e\., whether any content has been fabricated, such as CD1355; or whether any items are overly broad, such as simply stating "molecular testing" without specifying which particular test\)\. Additionally, these examination items should be of the type typically used in pathology, such as specific immunohistochemical antibodies, special histochemical stains, or particular molecular tests\. They should not include tests from other departments \(such as imaging studies or blood tests\)\. If the provided examination items can effectively differentiate among the given diagnoses and contain no erroneous content, output 1\. If the provided examination items cannot differentiate among the given diagnoses, but contain no erroneous content, output 0\. If the provided examination items contain fabricated content or involve tests from other departments, output \-1\. If the given diagnoses contain obvious errors or are too broad, making it impossible to propose reasonable additional tests, also output \-1\. Output format requirement:1​\|0\|−1Examination items: … Differential diagnoses: …

Here is the RAGES prompt\.

\{tcblisting\}

colback=gray\!10, colframe=black, title=RAGES Prompt, fonttitle=, coltitle=black, breakable, listing only, enhanced, sharp cornersBased on the given information, after careful consideration, infer the possible result of each examination item\. The given information includes the final diagnosis, examination items, existing results \(if any\), and relevant knowledge \(if any\)\. Specifically, you need to:

0\. Only focus on the content that can produce definitive results\. 1\. First, check the "Existing Results" and record results that overlap with the examination\. The confidence level is 1\. 2\. Then, check the "Relevant Knowledge"\. First, determine whether the relevant knowledge pertains to the same disease as described in the "Final Diagnosis"\. If it is the same disease, then, based on this knowledge, infer the results of the remaining examination items\. The confidence level is 0\.8\. 3\. Retrieve your own knowledge and speculate on the results of the remaining items\. The confidence level is 0\.6\. 4\. Output the above results in the specified format\. The format is as follows: ExamRes: "Item 1": \("Result 1", Confidence Level 1\), "Item 2": \("Result 2", Confidence Level 2\)

Information provided: Final Diagnosis: … Examinations: … Existing Results: … Relevant Knowledge: …

Here is the SFT training prompt\.

\{tcblisting\}

colback=gray\!10, colframe=black, title=SFT Prompt Template, fonttitle=, coltitle=black, breakable, listing only, enhanced, sharp cornerssystem: You are Qwen, created by Alibaba Cloud\. You are a helpful assistant\. A conversation between User and Assistant\. The user asks a question, and the Assistant solves it\. The assistant first thinks about the reasoning process in the mind and then provides the user with the answer\. The reasoning process and answer are enclosed within <think\> </think\> and <answer\> </answer\> tags, respectively, i\.e\., <think\> reasoning process here </think\> <answer\> answer here </answer\>\.

user: I need you to act as a professional pathologist\. After carefully considering the given information, infer the possible differential diagnoses\. Then, based on these differential diagnoses, suggest additional information that needs to be provided to rule out certain possibilities\. Specifically: 1\. First, you need to carefully analyze the given information, which mainly includes case background information, previous examination items, morphological descriptions of pathological sections, etc\. Summarize the evidence points related to the diagnosis from this information\. 2\. Based on the given information, analyze what the possible differential diagnoses are and determine whether they are consistent with the given information\. Note: These differential diagnoses should be as broad and accurate as possible \(broad means considering less common diagnostic possibilities, and accurate means the listed differential diagnoses should not conflict with most of the background information\)\. 3\. According to the listed differential diagnoses, propose the further examination items\. You need to specify the exact antigen \- antibody, staining type, or molecular type\. If the existing information is sufficient to confirm a specific disease, only output that disease and leave the additional examination items blank\. 4\. Finally, summarize the possible differential diagnoses and the required additional examination items in a given format\. When summarizing the differential diagnoses, you need to rank the more likely diagnoses higher\. 5\. Output four sections in the specified format: "Case Analysis \- Differential Diagnosis Analysis \- Additional Examination Items \- Summary"\.

Format requirements: \#\# Case Analysis … \#\# Summary \*\*Differential Diagnoses\*\*:\\DiffListDifferential Diagnosis 1, … \*\*Further Examinations\*\*:\\ExamListExamination Item 1, …

Here is the case: <Case Information\>

assistant: …

user: Now the results of the further examinations have come out\. I need you to: 1\. First, check the "Case Information" and the "First\-round Diagnosis" to sort out the previous diagnostic chain of thought and related conclusions\. 2\. Then, check the "Results of Further Examinations"\. The additional test results may not fully match the items requested in the initial diagnosis\. Based on the available test results, you need to conduct further differential analysis, and give the final diagnosis\. Note: You are completely entitled to overturn the initial diagnostic approach and provide a diagnosis based on the current information after obtaining more data\. 3\. The final diagnosis must be output in the specified format, i\.e\.,D​i​a​g​n​o​s​i​s​N​a​m​eHere is the information: Results of Further Examinations: <Exam Results\>

assistant: …

Here are the evaluation prompts for diagnoses and examinations\.\{tcblisting\}colback=gray\!10, colframe=black, title=Evaluation Prompts, fonttitle=, coltitle=black, breakable, listing only, enhanced, sharp corners

user: I need you to act as a professional pathologist\. After careful consideration based on the given disease candidates and the true diagnosis, determine whether the true diagnosis \(or a close approximation\) is among the candidates and, if present, its position in the list\. If it is within the candidates, outputT​r​u​e\+ "Hit candidate content" \+ "Position of the hit content" at the end; otherwise, outputF​a​l​s​e\+ No hit \+ 0\.

Ground truth diagnosis: … Candidate diagnoses: …

assistant: …T​r​u​e\|F​a​l​s​e\+ … \+ <digital\>

user: I need you to assist me in determining whether some pathological content is reasonable\. I will provide you with a list of differential diagnosis diseases, a set of further examination results, and the ground truth diagnosis\. You need to determine: 1\. Based on the list of differential diagnosis diseases, judge whether the additional examination items are reasonable and record the unreasonable items; 2\. Based on the ground truth diagnosis, judge whether the further examination results are reasonable and record the incorrect results\.

\*\*Notes\*\*: 1\. When the additional examination items are "no need," both items can be directly considered reasonable\. 2\. When judging the plausibility of examination results, do not consider whether some results are omitted; only judge the reasonableness of the existing examination results\.

The information you need to use is as follows: \- Differential diagnosis: … \- Further examinations and results: … \- Ground truth diagnosis: …

After careful consideration, you need to summarize at the end of the output in the following format: 1\. Exam:T​r​u​e\|F​a​l​s​e,\\ListWrong Item 1, … 2\. Result:T​r​u​e\|F​a​l​s​e,\\ListWrong Item and Result 1, …

assistant: … 1\. Exam:T​r​u​e\|F​a​l​s​e,\\List… 2\. Result:T​r​u​e\|F​a​l​s​e,\\List…

### A\.8Supervised Fine\-tuning Details

FollowingHuanget al\.\([2025](https://arxiv.org/html/2607.02983#bib.bib27)\), we create an SFT dataset generated by DeepSeek\-R1 to activate the reasoning of existing LLMs on two\-turn diagnosis\.

In theinitial consultation, LLMs take clinical history and histological findings as input and think about possible differential diagnoses and appended examinations\. Instead of directly giving the differential list in the original cases as a guide, we adopt a post\-verification strategy\. We only provide the input and generate it several times for each case\. After gathering these trials, we use another LLM to judge whether the ground truth diagnosis appears in these trials, and retain the positive ones as SFT data\. We chose this more complex strategy for two reasons\. First, we do not expect LLMs to be bound by the original text, since different pathologists might derive different disease suspects from the same case\. Second, when a differential list is provided, LLMs might have hallucinations like direct references to the original results\.

Based on the verified trials, we use the RAGES method to simulate the appended results\. After collecting sufficient appended results, we can start thefollow\-up consultation\. We provide LLMs with the response in the first turn and the acquired further examination results, and ask them to produce a final diagnosis after careful thinking\. Also, we employ the post\-verification strategy and retain as SFT data those trajectories that propose the true diagnosis\.

A total of 925 training samples related to initial consultations and 623 follow\-up samples are constructed\. For SFT, we adopt Qwen2\.5\-32B\-Instruct\(Yanget al\.,[2024](https://arxiv.org/html/2607.02983#bib.bib31)\)as the base model\. We apply parameter\-efficient Low\-Rank Adaptation \(LoRA\)\(Huet al\.,[2022](https://arxiv.org/html/2607.02983#bib.bib32)\)and enable bf16 precision to optimize training with our curated dataset\. The training workflow is implemented using Llama\-Factory\(Zhenget al\.,[2024](https://arxiv.org/html/2607.02983#bib.bib33)\), and evaluation is conducted with vLLM\(Kwonet al\.,[2023](https://arxiv.org/html/2607.02983#bib.bib34)\)\. We use LoRA with default hyperparameters, asα=16\\alpha=16andr=8r=8\. The initial learning rate is set to5×10−55\\times 10^\{\-5\}with cosine decay, and training is run for 15 epochs\. The fine\-tuning is carried out on 8 A100 GPUs, and the entire process completes in approximately 12 hours\.

### A\.9Dataset Details

The training dataset comes from two sources:

- •DakaPathis a Chinese platform for pathological teaching and communication\. In addition to its extensive knowledge, DakaPath features a special section called Micro Lecture, which provides expert explanations of hundreds of real\-life cases\. We collected 373 raw explanations\.
- •Chinese Journal of Pathologyreports on advanced scientific research achievements and pathological diagnosis experience as case discussions\. Initially, we included 653 cases\.

The evaluation dataset consists of cases from:

- •Pathology Outlinesis a comprehensive platform for pathology communication\. It offers extensive knowledge across subspecialties and weekly case studies\. We collected 483 cases published before December 2024\.
- •Hans Popper Hepatopathology Society \(HPHS\)is a hepatopathology\-focused community that presents a noteworthy case every 3 to 4 months\. We included 37 cases published up to September 2024\.
- •Thein\-house datasetoriginates from internal examination materials used for resident education at Hospital X\. It includes 339 cases covering 11 major domains, including the endocrine system, reproductive system, respiratory system, central nervous system, skin, bone and soft tissue, etc\.

### A\.10Other Details

Here are the detailed hyperparameters used in the RL algorithm\. If not specified, a default value is adopted\.

Table 10:Hyperparameters used in RL\.
### A\.11Case Study

We first present a case generated with our RL model to demonstrate the whole process\.

Case 1: Hepatocellular carcinomaCase Information:A 65 year old man presented with a large mass in the floor of the mouth \(levels Ia to VI\) but mostly suprahyoid\. Ultrasound showed a vascular appearance and a normal appearing thyroid gland\. The patient also had a prior history of hepatocellular carcinoma, treated with TACE, with a rising serum alpha fetoprotein\.The core biopsy shows a trabecular pattern of tumor cells separated by tortuous sinusoid\-like vascular channels\. At high power, the pleomorphic tumor cells have well defined cytoplasmic boundaries, a hepatoid appearance with abundant pink cytoplasm, and prominent nucleoli\. Bile plugs are easily seen\.Ground Truth Diagnosis:Hepatocellular carcinomaInitial Prediction:First, based on the case information, the patient is a 65\-year\-old male whose main symptom is a large mass on the floor of the mouth, extending from level Ia to VI, but primarily located suprahyoid\. Ultrasound shows a vascular appearance of the mass, with a normal thyroid gland\. The patient has a history of hepatocellular carcinoma, has undergone TACE treatment, and has an elevated serum alpha\-fetoprotein \(AFP\)\.From the histological description, the tumor cells are arranged in a trabecular pattern, with tortuous sinusoid\-like vascular structures\. Under high magnification, the tumor cells are pleomorphic, with well\-defined cytoplasmic boundaries, a hepatoid appearance, abundant pink cytoplasm, and prominent nucleoli\. In addition, bile plugs are present\.Based on this information, we can consider the following possible differential diagnoses: Hepatocellular carcinoma, Cholangiocarcinoma, Metastatic cancer, Lymphoma, Acinar cell carcinoma, Metastatic hepatocellular carcinoma, Metastatic cholangiocarcinoma, Metastatic lymphoma, Metastatic acinar cell carcinoma, Metastatic neuroendocrine tumor\.Next, we re\-examine these diagnoses and exclude options with extremely low probability\. By analyzing the case information, we can see that the mass is located on the floor of the mouth, the patient has a history of hepatocellular carcinoma, and the serum AFP is elevated\. These clues suggest that hepatocellular carcinoma and cholangiocarcinoma are relatively high\-probability diagnoses\. Metastatic cancer, lymphoma, and acinar cell carcinoma are also possible diagnoses\. The possibility of neuroendocrine tumor is relatively low, as the case information does not mention neuroendocrine features\.After reflection, we find that hepatocellular carcinoma and cholangiocarcinoma are the most likely diagnoses, because the patient has a history of hepatocellular carcinoma and the mass exhibits features of both hepatocytes and bile duct cells\. Metastatic cancer, lymphoma, and acinar cell carcinoma are also possible diagnoses, but with relatively lower probability\. The possibility of neuroendocrine tumor is relatively low, as the case information does not mention neuroendocrine features\.In conclusion, the differential diagnoses we consider include: Hepatocellular carcinoma, Cholangiocarcinoma, Metastatic cancer, Lymphoma, Acinar cell carcinoma, Metastatic hepatocellular carcinoma, Metastatic cholangiocarcinoma, Metastatic lymphoma, Metastatic acinar cell carcinoma\.Differential Diagnoses: \\DiffList\{Hepatocellular carcinoma, Cholangiocarcinoma, Metastatic cancer, Lymphoma, Acinar cell carcinoma, Metastatic hepatocellular carcinoma, Metastatic cholangiocarcinoma, Metastatic lymphoma, Metastatic acinar cell carcinoma\}Additional Examination Items: \\ExamList\{Immunohistochemistry \(e\.g\., AFP, CK, Vimentin, CD3, CD20, ACCP, S100, CK19, CK7, CD11c, CD68, CD117, CD34, Ki\-67, etc\.\), Special histochemical stains \(e\.g\., PAS, AB\-PAS, etc\.\), Molecular testing \(e\.g\., gene expression analysis for RAS, KRAS, BCL2, BCL6, MALT1, etc\.\)\}Follow\-up Prediction:First, we review the initial round of diagnostics\. The first round considered several possibilities: hepatocellular carcinoma, cholangiocarcinoma, metastatic carcinoma, lymphoma, acinar epithelial cell carcinoma, metastatic hepatocellular carcinoma, metastatic cholangiocarcinoma, metastatic lymphoma, metastatic acinar epithelial cell carcinoma\. Based on the case information, the mass is located on the floor of the mouth, with a history of hepatocellular carcinoma and an elevated serum alpha\-fetoprotein \(AFP\)\. Histological description indicates that the tumor cells are arranged in a trabecular pattern, with tortuous sinusoid\-like vascular structures\. Under high magnification, the tumor cells appear pleomorphic, with well\-defined cytoplasmic boundaries, a hepatoid appearance, abundant pink cytoplasm, and prominent nucleoli\. Additionally, bile plugs are present\.Next, we examine the results of the additional tests: AFP: Positive CK: Positive Vimentin: Negative CD3: Negative CD20: Negative ACCP: Positive S100: Negative CK19: Negative CK7: Negative CD11c: Negative CD68: Negative CD117: Negative CD34: Positive Ki\-67: 20% Positive RAS: Wild\-type KRAS: Wild\-type BCL2: Negative BCL6: Negative MALT1: NegativeBased on these results, we can further analyze:Both AFP and ACCP are positive, suggesting possible hepatocellular carcinoma or cholangiocarcinoma\.Vimentin, CD3, and CD20 are all negative, suggesting the tumor is not of mesenchymal or lymphoid origin\.CD34 is positive, suggesting a possible epithelial origin\.Ki\-67 is 20% positive, indicating moderate tumor cell proliferation activity\.Both RAS and KRAS are wild\-type, suggesting the tumor does not harbor certain specific gene mutations\.BCL2, BCL6, and MALT1 are all negative, suggesting it is not lymphoma or MALT lymphoma\.Synthesizing this information, we can narrow the diagnostic possibilities to:Hepatocellular carcinoma Cholangiocarcinoma Acinar epithelial cell carcinoma Metastatic hepatocellular carcinoma Metastatic cholangiocarcinoma Metastatic acinar epithelial cell carcinomaThrough analysis, we find that the positivity of both AFP and ACCP suggests hepatocellular carcinoma or cholangiocarcinoma\. CD34 positivity suggests an epithelial origin\. The 20% Ki\-67 positivity indicates moderate proliferative activity\. Wild\-type RAS and KRAS suggest the absence of specific mutations\. The negativity of BCL2, BCL6, and MALT1 rules out lymphoma or MALT lymphoma\.In conclusion, our final diagnostic considerations include: Hepatocellular carcinoma Cholangiocarcinoma Acinar epithelial cell carcinoma Metastatic hepatocellular carcinoma Metastatic cholangiocarcinoma Metastatic acinar epithelial cell carcinomaFinal Diagnosis: \\boxed\{Hepatocellular carcinoma\}

We then present a case generated with our RL model, which produces a vague diagnosis in the initial turn and refines it into a precise one in the second turn\.

Case 2: Diagnosis RefinementCase Information:An HIV seronegative, 96 year old African\-American woman with no remarkable past medical history presented with a left upper eyelid lesion, which was completely excised\. The specimen consisted of skin and mucosa\-covered soft tissue\. Immediately subjacent to the hairy edge was a 0\.7 x 0\.5 x 0\.5 cm indurated nodule, with a mottled brown tan cut surface\. Histological assessment showed a circumscribed stromal tumor, with spindle cells forming vascular spaces, extravasated red blood cells and occasional hyaline globules\.Ground Truth Diagnosis:Kaposi sarcomaInitial Prediction:First, based on the case information, the patient is a 96\-year\-old African American female with no significant past medical history, presenting with a lesion on the left upper eyelid\. The pathological findings reveal a well\-circumscribed stromal tumor composed of spindle cells forming vascular spaces, accompanied by red blood cell extravasation and occasional hyaline bodies\.Initial Differential Diagnoses: Fibroma, Angiofibroma, Metastatic tumor, Metastatic melanoma, Metastatic neuroendocrine tumor, Metastatic lymphoma, Metastatic lipoma, Metastatic leiomyoma, Metastatic epithelial tumor, Metastatic mesenchymal tumor Reflection and Adjustment Fibroma: Although spindle cells form vascular spaces, fibromas typically do not exhibit red blood cell extravasation or hyaline bodies\.Angiofibroma: While vascular spaces and red blood cell extravasation are present, hyaline bodies are not usually associated with this diagnosis\.Metastatic tumor: Cannot be completely ruled out, but requires more evidence for confirmation\.Metastatic melanoma: Cannot be completely ruled out, but requires more evidence for confirmation\.Metastatic neuroendocrine tumor: Cannot be completely ruled out, but requires more evidence for confirmation\.Metastatic lymphoma: Cannot be completely ruled out, but requires more evidence for confirmation\.Metastatic lipoma: Cannot be completely ruled out, but requires more evidence for confirmation\.Metastatic leiomyoma: Cannot be completely ruled out, but requires more evidence for confirmation\.Metastatic epithelial tumor: Cannot be completely ruled out, but requires more evidence for confirmation\.Metastatic mesenchymal tumor: Cannot be completely ruled out, but requires more evidence for confirmation\.Final Differential Diagnoses Fibroma, Angiofibroma, Metastatic tumor, Metastatic melanoma, Metastatic neuroendocrine tumor, Metastatic lymphoma, Metastatic lipoma, Metastatic leiomyoma, Metastatic epithelial tumor, Metastatic mesenchymal tumorAdditional Examination Items Immunohistochemistry \(e\.g\., Vimentin, CD34, CD3, CD20, S100, CK, CD30, CD11c, CD68, fibroblastic markers, angiogenic markers, etc\.\)Special histochemical stains \(e\.g\., PAS, AB\-PAS, etc\.\)Molecular testing \(e\.g\., FISH, PCR, etc\.\)Summary Differential Diagnoses: \\DiffList\{Fibroma, Angiofibroma, Metastatic tumor, Metastatic melanoma, Metastatic neuroendocrine tumor, Metastatic lymphoma, Metastatic lipoma, Metastatic leiomyoma, Metastatic epithelial tumor,Metastatic mesenchymal tumor\}Additional Examination Items: \\ExamList\{Immunohistochemistry \(e\.g\., Vimentin, CD34, CD3, CD20, S100, CK, CD30, CD11c, CD68, fibroblastic markers, angiogenic markers, etc\.\), Special histochemical stains \(e\.g\., PAS, AB\-PAS, etc\.\), Molecular testing \(e\.g\., FISH, PCR, etc\.\)\}Follow\-up Prediction:Re\-evaluating the Initial DiagnosisBased on the initial diagnosis, we considered the following possibilities: Fibroma, Angiofibroma, Metastatic tumor, Metastatic melanoma, Metastatic neuroendocrine tumor, Metastatic lymphoma, Metastatic lipoma, Metastatic leiomyoma, Metastatic epithelial tumor, Metastatic mesenchymal tumor\.Analyzing the Additional Test ResultsVimentin: Positive CD34: Positive S100: Negative CK: Negative CD3: Negative CD20: Negative CD30: Negative CD11c: Negative CD68: Negative PCR: Positive \(for HHV\-8\) FISH: Positive \(for HHV\-8\)Further AnalysisThe positivity of both Vimentin and CD34 suggests a mesenchymal origin of the tumor\.The negativity of both S100 and CK suggests it is not a neuroendocrine or epithelial tumor\.The negativity of CD3, CD20, CD30, CD11c, and CD68 suggests it is not lymphoma\.The detection of HHV\-8 by both PCR and FISH suggests a possible association with Kaposi sarcoma\.Final DiagnosisBased on the additional test results, the most likely diagnosis is Kaposi sarcoma\. Kaposi sarcoma is a mesenchymal tumor caused by HHV\-8, characterized by spindle cells forming vascular spaces, accompanied by red blood cell extravasation and hyaline bodies\. The results of Vimentin and CD34 positivity, S100 and CK negativity, and the detection of HHV\-8 by both PCR and FISH all support the diagnosis of Kaposi sarcoma\.Output Final Diagnosis\\boxed\{Kaposi sarcoma\}

### A\.12Extending to More Turns

In the main manuscript, our research centers on a two\-turn diagnostic framework\. However, the proposed method can naturally extend to interactions with more turns, provided that all intermediate stages remain evidence\-based, i\.e\., they do not involve subjective outcomes such as treatment decisions\.

Here, we present a representativethree\-turndiagnostic process for a case of MALT lymphoma\. In this example, the additional clinical evidence is divided into two sequential stages: immunohistochemistry \(IHC\) results first, followed by molecular testing results\. Across these three turns of interaction, the diagnostic specificity progressively increases from “lymphoma \(especially bronchial lymphoma\),” to “B\-cell\-derived lymphoma \(particularly MALT lymphoma\),” and ultimately to the final diagnosis of “MALT lymphoma\.”

Case 3: MALT lymphoma \(Give IHC results first, then the molecular testing\)## System: You are Qwen, created by Alibaba Cloud\. You are a helpful assistant\. A conversation between User and Assistant\. The user asks a question, and the Assistant solves it\. The assistant first drafts the reasoning process \(inner monologue\) until it has derived the final answer with full confidence\. It then provides a self\-contained summary of the thoughts, i\.e\., keeping succinct but containing all the critical steps needed to reach the conclusion\. It should use Markdown and Latex to format the response\. Write both the thoughts and summary in the same language as the task posed by the user\. The thinking process must follow the template below \(You should \*\*include and only include one\*\* pair of <think\></think\>and <answer\></answer\>tags in your response\): <think\>The thoughts or/and draft, like working through an exercise on scratch paper\. Be as casual and as long as necessary until it is confident to generate a correct answer\. </think\> <answer\>Here, provide a concise summary that reflects the reasoning process and presents a clear final answer to the user\. </answer\> ## User 1: I need you to infer possible differential diagnoses and related additional tests based on the provided case information\. Specifically, you need to analyze the given case information carefully\. Then, based on the evidence points within it, gradually deduce all possible related differential diagnoses \(these diagnoses must be relatively specific, avoiding simple descriptions such as "benign/malignant lesion"\)\. Then, re\-examine the case information to exclude those differential options with extremely low probability \(actively reflect: does the existing information contain anything that clearly conflicts with this disease?\)\. At the same time, you may also add new differential diagnoses \(actively reflect: have I missed any possibilities?\)\. You can engage in such reflection even during the initial inference\. After repeating this process for multiple rounds, when you are sufficiently confident that the current set of differential diagnoses has a high probability of including the final diagnosis while having an extremely low likelihood of including unrelated diagnoses, you may proceed to the final summary stage\. When summarizing the final differential diagnoses, rank those with a higher probability first\. Note that these differential diagnoses should not exceed ten\. Then, based on these differential diagnoses, provide the additional information needed to rule out some of the possible diagnoses \(such as a specific immunohistochemical antibody, a specialized histochemical stain, or a particular molecular test\)\. While thinking, you may review previous content at any time for reflection and promptly revise your conclusions as needed\. After thinking, you need to organize your response: briefly summarize your thought process, then summarize your output in the specified format\. Format requirements are as follows: Differential Diagnoses: \\DiffList\{Diagnosis 1, Diagnosis 2, …\} Further Examination Items: \\ExamList\{Item 1, Item 2, …\} The following is the case information: Male, 74 years old, presented with cough and shortness of breath\. Shortness of breath occurs when climbing stairs or walking briskly\. Chest CT at an outside hospital showed enlargement of the right pulmonary hilum, high\-density lesions in the right main bronchus and some segmental bronchi, and patchy high\-density shadows in the right lung\. Pulmonary function test revealed severe mixed ventilatory dysfunction\. Bronchoscopy showed irregular mucosal protrusions in the right main bronchus, upper lobe, and intermediate bronchus, causing significant luminal narrowing; a biopsy was taken from this area\. Morphological features: proliferation of small, atypical monocytoid lymphocytes in the submucosa of the bronchial mucosa, with scattered immunoblasts or centroblasts present; tumor cells infiltrated the bronchial mucosal epithelium, forming lymphoepithelial lesions\. ## Assistant 1: <think\> First, based on the case information, the patient is a 74\-year\-old male presenting primarily with cough and shortness of breath\. Chest CT showed enlargement of the right pulmonary hilum, high\-density lesions in the right main bronchus and some segmental bronchi, and patchy high\-density shadows in the right lung\. Pulmonary function tests revealed severe mixed ventilatory dysfunction\. Bronchoscopy showed irregular mucosal protrusions in the right main bronchus, upper lobe, and intermediate bronchus, causing significant luminal narrowing\. Biopsy revealed proliferation of small, atypical monocytoid lymphocytes in the submucosa of the bronchial mucosa, with scattered immunoblasts or centroblasts; tumor cells infiltrated the bronchial mucosal epithelium, forming lymphoepithelial lesions\. Based on this information, we can consider the following possible differential diagnoses: 1\. Lymphoma \(especially bronchial lymphoma\) 2\. Adenocarcinoma 3\. Sarcoma 4\. Fibrous tumor 5\. Lymphocytosis 6\. Mucinous tumor 7\. Metastatic carcinoma 8\. Inflammatory diseases \(e\.g\., interstitial lung disease\) 9\. Benign hyperplasia \(e\.g\., adenoma\) 10\. Lymphoepithelial lesions Next, we re\-evaluate these diagnoses and exclude those with very low probability\. By analyzing the case information, we find that the submucosal proliferation of small, atypical monocytoid lymphocytes with scattered immunoblasts or centroblasts, along with tumor cell infiltration of the bronchial mucosal epithelium forming lymphoepithelial lesions, suggests lymphoma or an inflammatory disease\. Adenocarcinoma, sarcoma, fibrous tumor, and mucinous tumor are less likely because the case does not show obvious features of these conditions\. Benign hyperplasia is also unlikely due to the presence of clearly abnormal cell proliferation\. The possibility of metastatic carcinoma cannot be completely excluded, as a tumor from another site could have metastasized to the lung\. Therefore, we can narrow down the differential diagnosis to the following more likely possibilities: 1\. Lymphoma \(especially bronchial lymphoma\) 2\. Inflammatory diseases \(e\.g\., interstitial lung disease\) 3\. Lymphocytosis 4\. Metastatic carcinoma 5\. Lymphoepithelial lesions Upon reflection, we note that lymphoepithelial lesions represent a specific diagnosis that may result from abnormal proliferation of lymphocytes and epithelial cells\. Therefore, we can further narrow the differential diagnosis to the following likely possibilities: 1\. Lymphoma \(especially bronchial lymphoma\) 2\. Inflammatory diseases \(e\.g\., interstitial lung disease\) 3\. Lymphocytosis 4\. Metastatic carcinoma 5\. Lymphoepithelial lesions </think\> <answer\> Differential diagnosis:\\DiffList\{Lymphoma \(especially bronchial lymphoma\), Inflammatory diseases \(e\.g\., interstitial lung disease\), Lymphocytosis, Metastatic carcinoma, Lymphoepithelial lesions\} Further Examination Items:\\ExamList\{Immunohistochemistry \(e\.g\., CD3, CD20, CK, S100, Vimentin, MUC1, MUC5AC, MUC6, CD11c, CD68, CD23, CD15, CD11b, CD1a, CD30, CD138, etc\.\), special stains \(e\.g\., PAS, AB\-PAS, etc\.\), molecular testing \(e\.g\., gene expression analysis of BCL2, BCL6, MALT1, TP53, KRAS, NRAS, BRAF, etc\.\)\} </answer\> ## User 2: Now some of the additional test results are available\. I need you to: 1\. First review the “case information” and the “first\-round diagnosis” to clarify the previous diagnostic reasoning and conclusions\. 2\. Then review the “additional test results\.” The additional results may not fully correspond to the extra tests requested during the first\-round diagnosis\. Based on the available results, you should further refine the differential diagnosis on top of the first\-round assessment, and provide both the diagnosis and any additional tests that should be performed next\. Note: after receiving more information, you may completely overturn the first\-round reasoning and provide a diagnosis based solely on the current evidence\. 3\. The final output format must follow: Differential Diagnosis:\\DiffList\{Diagnosis 1, Diagnosis 2, …\} Further Examination Items:\\ExamList\{Item 1, Item 2, …\} Additional Test Results CD3: Negative CD20: Positive CK: Negative S100: Negative Vimentin: Positive MUC1: Negative MUC5AC: Negative MUC6: Negative CD11c: Positive CD68: Positive CD23: Negative CD15: Negative CD11b: Negative CD1a: Negative CD30: Negative CD138: Positive ## Assistant 2: <think\>First, we review the diagnostic reasoning and conclusions from the first round\. The initial diagnostic considerations included the following relatively likely possibilities: lymphoma \(especially bronchial lymphoma\), inflammatory diseases \(such as interstitial lung disease\), lymphocytosis, metastatic carcinoma, and lymphoepithelial lesions\. According to the morphological description, there is proliferation of mononuclear, atypical, clear small lymphocytes beneath the bronchial mucosa, with scattered immunoblasts or centroblasts among them; tumor cells infiltrate the bronchial mucosal epithelium, forming lymphoepithelial lesions\. These findings suggest the possibility of lymphoma or inflammatory disease\. Adenocarcinoma, sarcoma, fibrous tumors, mucinous tumors, and other such entities are relatively less likely because the case information does not strongly support these features\. Benign hyperplasia is also unlikely due to the presence of clearly abnormal cellular proliferation\. Metastatic carcinoma cannot be entirely ruled out, as tumors from other sites may metastasize to the lung\. Therefore, we can narrow the diagnostic range to the following relatively likely diagnoses: lymphoma \(especially bronchial lymphoma\), inflammatory diseases \(such as interstitial lung disease\), lymphocytosis, metastatic carcinoma, and lymphoepithelial lesions\. Next, we examine the additional test results\. These include immunohistochemical staining and special staining results\. The specific results are as follows: \- CD3: negative \- CD20: positive \- CK: negative \- S100: negative \- Vimentin: positive \- MUC1: negative \- MUC5AC: negative \- MUC6: negative \- CD11c: positive \- CD68: positive \- CD23: negative \- CD15: negative \- CD11b: negative \- CD1a: negative \- CD30: negative \- CD138: positive Based on these results, we can analyze further\. CD20 and CD138 positivity suggests a B\-cell\-derived neoplasm, such as diffuse large B\-cell lymphoma or MALT lymphoma\. CD3 negativity, CD20 positivity, and CD138 positivity point toward B\-cell lymphoma\. CD68 positivity suggests a histiocytic component\. CD11c and CD68 positivity indicates possible histiocytic or macrophage lineage\. CD23 negativity, CD15 negativity, CD11b negativity, CD1a negativity, and CD30 negativity suggest a non\-Hodgkin lymphoma rather than Hodgkin lymphoma\. The negativity of MUC1, MUC5AC, and MUC6 suggests a non\-mucinous tumor\. Taken together, the most likely diagnosis is a B\-cell\-derived lymphoma, particularly MALT lymphoma\. Based on this new information, we can further narrow the differential diagnosis to the following relatively likely possibilities: B\-cell\-derived lymphoma \(particularly MALT lymphoma\), inflammatory diseases \(such as interstitial lung disease\), and metastatic carcinoma\. Other less likely diagnoses need not be considered further, as the new test results provide more information\. </think\> <answer\> Differential diagnosis:\\DiffList\{B\-cell\-derived lymphoma \(particularly MALT lymphoma\), inflammatory diseases \(such as interstitial lung disease\), metastatic carcinoma\} Further Examination Items:\\ExamList\{Molecular testing \(such as BCL2, BCL6, MALT1, TP53, KRAS, NRAS, BRAF and other gene expression analyses\)\} </answer\> ## User 3: Now the results of the molecular testing are available\. I need you to: 1\. First review the “case information" and the previous rounds of diagnoses to clarify the earlier diagnostic reasoning and related conclusions\. 2\. Then review this round of “additional test results\." The additional test results may not fully match the extra tests that were previously requested\. Based on the available results, you should further analyze the differential diagnosis on top of the previous conclusions and provide the final diagnosis\. Note: after receiving new information, you may completely overturn the prior diagnostic reasoning and provide the diagnosis supported by the current data\. 3\. The final diagnosis must be output in the specified format, e\.g\.: \\boxed\{Diagnosis\} Additional Test Results: BCL2: no rearrangement BCL6: no rearrangement MALT1: rearrangement positive TP53: no mutation KRAS: no mutation NRAS: no mutation BRAF: no mutation ## Assistant 3: <think\>First, we review the diagnostic reasoning and conclusions from the previous rounds\. The earlier differential diagnosis focused on the following most likely possibilities: B‐cell lymphoma \(particularly MALT lymphoma\), inflammatory diseases \(such as interstitial lung disease\), and metastatic carcinoma\. Based on the morphological description and immunohistochemical findings\-namely, the proliferation of monocytoid, atypical clear small lymphocytes in the bronchial submucosa, scattered immunoblasts or centroblasts, and tumor cell infiltration of the bronchial mucosal epithelium forming lymphoepithelial lesions—these features suggested lymphoma or inflammatory disease\. Among them, B‐cell lymphoma, especially MALT lymphoma, was the most likely diagnosis\. CD20 and CD138 positivity indicated a B‐cell neoplasm such as MALT lymphoma\. CD68 positivity suggested a possible histiocytic component\. CD11c and CD68 positivity indicated histiocytes or macrophages\. CD23 negativity, CD15 negativity, CD11b negativity, CD1a negativity, and CD30 negativity supported non–Hodgkin lymphoma rather than Hodgkin lymphoma\. The negativity for MUC1, MUC5AC, and MUC6 indicated a non‐mucinous tumor\. Next, we review the additional test results from this round\. These included molecular findings\. The specific results are as follows: \- BCL2: no rearrangement \- BCL6: no rearrangement \- MALT1: rearrangement positive \- TP53: no mutation \- KRAS: no mutation \- NRAS: no mutation \- BRAF: no mutation Based on these results, we can further analyze the diagnosis\. The absence of BCL2 and BCL6 rearrangements suggests a lymphoma without BCL2 or BCL6 rearrangements\. A positive MALT1 rearrangement strongly supports MALT lymphoma\. The absence of TP53, KRAS, NRAS, and BRAF mutations suggests a lymphoma without these genetic mutations\. In summary, the most likely diagnosis is MALT lymphoma\. Based on this new information, we can determine that the final diagnosis is MALT lymphoma\. </think \> <answer \>\\boxed\{MALT lymphoma\} </answer \>