GuideSkill: Evolving Executable LLM Agent Skills for Guideline-Grounded Clinical Reasoning

# Evolving Executable LLM Agent Skills for Guideline-Grounded Clinical Reasoning
Source: [https://arxiv.org/html/2607.26160](https://arxiv.org/html/2607.26160)
###### Abstract

Clinical practice guidelines \(CPGs\) encode diagnostic criteria, but LLM systems typically retrieve guideline text or absorb it through training rather than execute its rules\. We introduceGuideSkill, an external reasoning layer that compiles disease\-specific criteria into executable functions returning ordinal diagnostic\-support scores\.GuideSkill\-Zerois initialized from guidelines, whileGuideSkill\-Evouses case–diagnosis pairs to refine covered skills and add missing diagnoses\. At inference, an LLM proposes a differential diagnosis, grounds the features required by each matched skill, and fuses its ranking with the executed skill scores\. Across four benchmarks and four backbones,GuideSkill\-Zeroimproves macro\-average accuracy over guideline RAG by 13\.45% on average\.GuideSkill\-Evoachieves the highest macro\-average for every backbone, improves over direct inference by 18\.49% relatively, and increases gold\-label skill coverage from 56\.5% to 99\.5%\. On Qwen3\.5\-9B, it also exceeds the strongest parameter\-update baseline by 11\.16% without updating the backbone\. Expert evaluation further indicates thatGuideSkillproduces clinically sound and broadly acceptable skills, suggesting that its initialized and evolved rules are reliable and practically meaningful\. These results support executable skills as a model\-agnostic mechanism for combining guideline\-derived procedures with case\-derived diagnostic patterns\.

## 1Introduction

Clinical diagnosis is not only a knowledge\-recall problem\. It is a multistep process that requires gathering and synthesizing patient information, generating and comparing plausible diagnoses, determining which findings and thresholds support or weaken each candidate, excluding alternatives, and identifying decisive tests\(McDuffet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib2); Hageret al\.[2024](https://arxiv.org/html/2607.26160#bib.bib4); Caoet al\.[2026](https://arxiv.org/html/2607.26160#bib.bib13); Youet al\.[2026](https://arxiv.org/html/2607.26160#bib.bib37)\)\. Clinical practice guidelines \(CPGs\) encode evidence\-based recommendations and conditional decision logic that can guide these decisions\(Institute of Medicine[2011](https://arxiv.org/html/2607.26160#bib.bib38); Shenet al\.[2026b](https://arxiv.org/html/2607.26160#bib.bib21)\)\. However, providing guideline\-derived criteria to an LLM does not guarantee that they will be applied correctly to the patient: even when guideline\-based checklists are supplied, criterion\-level evaluation remains imperfect\(Schubertet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib16)\)\. LLMs can generate ranked differential diagnoses from free\-text cases\(McDuffet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib2)\), but may omit clinician\-identified reasoning evidence and fail to follow diagnostic guidelines\(Wuet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib1); Hageret al\.[2024](https://arxiv.org/html/2607.26160#bib.bib4)\)\. We therefore ask whether CPGs can be transformed from passive references into an evolvable library of executable diagnostic skills that combines flexible candidate generation with explicit disease\-specific rule application\.

Prior work incorporates CPGs by supplying guideline text or checklists at inference\(Schubertet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib16)\), adapting model parameters with guideline\-containing corpora or guideline\-derived supervision\(Chenet al\.[2023](https://arxiv.org/html/2607.26160#bib.bib8); Stanieket al\.[2025](https://arxiv.org/html/2607.26160#bib.bib15); Shenet al\.[2026b](https://arxiv.org/html/2607.26160#bib.bib21)\), and translating guidelines into structured decision trees or program\-aided pathways\(Onianiet al\.[2024](https://arxiv.org/html/2607.26160#bib.bib18); Liet al\.[2023](https://arxiv.org/html/2607.26160#bib.bib20); Denget al\.[2026](https://arxiv.org/html/2607.26160#bib.bib17)\)\. These approaches demonstrate several ways to operationalize guideline knowledge in LLM systems, but they leave open a complementary systems question: can guideline\-derived procedures be organized as an external, disease\-indexed skill library that compares evidence across candidate diagnoses, transfers across LLM backbones, and expands without updating model parameters? This question is particularly relevant to differential diagnosis, where several conditions may plausibly explain the same presentation and must be compared against disease\-specific evidence\(McDuffet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib2)\)\. Because the initial guideline corpus covers only a finite set of diagnoses, we further examine whether labeled cases can extend the library to diagnoses absent from that corpus, including the rare and complex conditions represented in case\-report benchmarks\(Wuet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib1)\)\.

We introduceGuideSkill, which transforms CPGs from passive references into an external library of executable, disease\-specific skills\. Externalizing this knowledge makes diagnostic criteria reusable across LLM backbones and allows the library to be updated without modifying model parameters\.GuideSkill\-Zeroinitializes the library from guidelines, providing explicit supporting and contradictory rules for covered diagnoses\.GuideSkill\-Evothen uses labeled cases to refine existing skills and add diagnoses absent from the guideline corpus, expanding coverage while updating disease\-specific decision rules outside the backbone\. At inference, the LLM and the skill library address complementary limitations\. The LLM generates a ranked differential from the full patient narrative, including diagnoses beyond the finite skill library, while matched skills apply explicit disease\-specific criteria to covered candidates\. Their fusion preserves open\-ended candidate generation while allowing reusable evidence rules to refine the ranking, rather than relying entirely on either an LLM\-only ranking or incomplete skill coverage\. Because the skills remain external, the same diagnostic logic can be inspected, updated, and reused across backbones\.

We evaluateGuideSkillon four heterogeneous diagnostic\-reasoning benchmarks—MedCaseReasoning\(Wuet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib1)\), ER\-Reason\(Mehandruet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib5)\), MIMIC\-CDM\-FI\(Hageret al\.[2024](https://arxiv.org/html/2607.26160#bib.bib4)\), and MedThink\-Bench\(Zhouet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib35)\)—using four proprietary and open\-weight LLM backbones\. Using only guideline\-derived executable skills,GuideSkill\-Zeroachieves higher macro\-average accuracy than guideline RAG for every backbone\. After evolution,GuideSkill\-Evooutperforms direct inference in all 16 dataset–backbone comparisons, with a mean relative improvement of 18\.49%, while increasing pooled gold\-label skill coverage from 56\.50% to 99\.50%\. On MedThink\-Bench, which is excluded from evolution training, it achieves the best or tied\-best accuracy across all four backbones, demonstrating transfer to an unseen benchmark\. On Qwen3\.5\-9B, it also outperforms the evaluated supervised fine\-tuning, reinforcement\-learning, and guideline\-decision\-tree baselines without updating the backbone\. Together, these results demonstrate the benefits of external executable skills for diagnostic accuracy and coverage\.

Our contributions are threefold:

- •We formulate guideline\-grounded diagnosis as agentic skill execution and develop a disease\-indexed compilation pipeline that converts CPG recommendations into executable ordinal scorers\.
- •We introduce a case\-conditioned evolution mechanism that refines covered skills and adds previously uncovered diagnoses outside the LLM parameters\.
- •We develop a candidate\-level inference procedure and evaluate it across four backbones and four benchmarks, separating the contribution of guideline\-only initialization from the additional coverage and downstream gains obtained through evolution\.

## 2Related Work

Clinical Guidelines as Reasoning Resources\.Because CPGs synthesize reviewed clinical evidence into recommendations, they are a valuable foundation for clinical decision support\(Institute of Medicine[2011](https://arxiv.org/html/2607.26160#bib.bib38)\)\. LLM\-based systems have incorporated them as retrieved prompt context\(Schubertet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib16); Onianiet al\.[2024](https://arxiv.org/html/2607.26160#bib.bib18)\), supervised examples\(Stanieket al\.[2025](https://arxiv.org/html/2607.26160#bib.bib15)\), or reinforcement\-learning signals\(Tziakouri and Menolascina[2025](https://arxiv.org/html/2607.26160#bib.bib19)\)\. More structured approaches operationalize their decision logic: MedDM represents clinical pathways as LLM\-executable guidance trees\(Liet al\.[2023](https://arxiv.org/html/2607.26160#bib.bib20)\), CPGPrompt translates CPGs into decision trees traversed during inference\(Denget al\.[2026](https://arxiv.org/html/2607.26160#bib.bib17)\), and MedGuideX executes guideline logic to generate factual and counterfactual supervision for post\-training\(Shenet al\.[2026b](https://arxiv.org/html/2607.26160#bib.bib21)\)\. However, the guideline\-derived component of these systems remains bounded by the source corpus: it provides no procedure for diagnoses outside that corpus and no mechanism to learn additional diagnostic patterns from labeled case presentations\.GuideSkilladdresses this gap by initializing an external skill library from guidelines and using labeled cases to refine covered skills and add uncovered diagnoses\. It thereby retains guideline\-derived procedures while extending beyond the initial corpus\.

Evolving Skills in Healthcare and Biomedicine\.Agent skills externalize reusable procedures so that capabilities can accumulate without changing model parameters\. Trace2Skill induces transferable operating procedures from recurring patterns in execution trajectories\(Niet al\.[2026](https://arxiv.org/html/2607.26160#bib.bib22)\), while SkillClaw aggregates cross\-user trajectories to refine and extend a shared skill repository\(Maet al\.[2026](https://arxiv.org/html/2607.26160#bib.bib25)\)\. In healthcare, an empirical study of public skills finds that they primarily support patient\-facing workflow automation and monitoring, with limited coverage of diagnosis and treatment\(Xuet al\.[2026](https://arxiv.org/html/2607.26160#bib.bib27)\)\. Related biomedical systems target scientific workflows: SkillFoundry mines heterogeneous resources into validated executable skills and demonstrates them on genomics tasks\(Shenet al\.[2026a](https://arxiv.org/html/2607.26160#bib.bib53)\), whereas STELLA evolves reasoning templates and tool use for biomedical research and experimental discovery\(Jinet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib28)\)\. These studies capture experience or operational procedures but do not address evolving evidence\-based knowledge for patient\-level diagnosis\.GuideSkillfills this gap by initializing skills from CPGs, expanding them with labeled cases, and executing them against patient evidence\.

![Refer to caption](https://arxiv.org/html/2607.26160v1/x1.png)Figure 1:Overview ofGuideSkill\. Guideline recommendations are compiled into the initial executable library \(GuideSkill\-Zero\); labeled cases refine covered skills and add missing diagnoses \(GuideSkill\-Evo\); and, during inference, executed skill scores are fused with an LLM\-generated differential ranking\.
## 3Methodology

Figure 1 provides an overview ofGuideSkill\. The framework first constructs an initial skill library,GuideSkill\-Zero, from clinical guidelines, then evolves the library with real patient cases to obtainGuideSkill\-Evo, and finally applies the learned disease\-specific skills during inference\.

### 3\.1Task Formulation

Given an undiagnosed patient casexx, our goal is to predict the final diagnosisd⋆∈𝒟d^\{\\star\}\\in\\mathcal\{D\}\. Here,xxincludes the patient’s demographics, chief complaint, medical history, physical examination findings, and available test results\. The diagnosis space𝒟\\mathcal\{D\}is standardized using three\-character WHO ICD\-10 categories\(World Health Organization[2019](https://arxiv.org/html/2607.26160#bib.bib39)\), which provide normalized disease identifiers across datasets\.

During inference, an LLMMMfirst generates a ranked differential diagnosis set:

𝒞​\(x\)=\{d1,…,dK\},𝒞​\(x\)⊆𝒟,\\mathcal\{C\}\(x\)=\\\{d\_\{1\},\\dots,d\_\{K\}\\\},\\quad\\mathcal\{C\}\(x\)\\subseteq\\mathcal\{D\},\(1\)whereK=5K=5by default\. For each candidate diagnosisd∈𝒞​\(x\)d\\in\\mathcal\{C\}\(x\), we convert its rank into an LLM ranking scoresLLM​\(x,d\)s\_\{\\mathrm\{LLM\}\}\(x,d\)\. Specifically, ifddis ranked at positionrdr\_\{d\}with zero\-based indexing, then

sLLM​\(x,d\)=1rd\+1\.s\_\{\\mathrm\{LLM\}\}\(x,d\)=\\frac\{1\}\{r\_\{d\}\+1\}\.\(2\)
GuideSkillthen retrieves the corresponding disease\-specific skillgdg\_\{d\}from the skill library𝒢\\mathcal\{G\}and executes it on the patient case to obtain a skill\-based evidence score:

sskill​\(x,d\)=gd​\(x\)\.s\_\{\\mathrm\{skill\}\}\(x,d\)=g\_\{d\}\(x\)\.\(3\)The final diagnosis score combines the LLM ranking score and the skill\-based evidence score:

S​\(x,d\)=α​sLLM​\(x,d\)\+\(1−α\)​sskill​\(x,d\),S\(x,d\)=\\alpha s\_\{\\mathrm\{LLM\}\}\(x,d\)\+\(1\-\\alpha\)s\_\{\\mathrm\{skill\}\}\(x,d\),\(4\)whereα∈\[0,1\]\\alpha\\in\[0,1\]controls the relative contribution of the two signals\. The final prediction is selected from the candidate set:

d^=arg⁡maxd∈𝒞​\(x\)⁡S​\(x,d\)\.\\hat\{d\}=\\arg\\max\_\{d\\in\\mathcal\{C\}\(x\)\}S\(x,d\)\.\(5\)

### 3\.2Skill Initialization

The first stage constructs the initial skill library, denoted asGuideSkill\-Zero\. Given a collection of clinical practice guidelines,𝒫=\{p1,…,pM\},\\mathcal\{P\}=\\\{p\_\{1\},\\dots,p\_\{M\}\\\},we first filter and curate the raw guideline documents to obtain a high\-quality guideline set𝒫′\\mathcal\{P\}^\{\\prime\}\. Details of the guideline preprocessing are provided in Appendix B\. Each curated guideline is then parsed into disease\-specific clinical recommendations, where each recommendation captures a clinically relevant diagnostic criterion, finding, exclusion rule, or decision rule that supports or refutes a specific diagnosis\.

SinceGuideSkilloperates at the ICD\-10 category level, we group recommendations by their mapped ICD\-10 diagnosis category\. For a diagnosis categorydd, we denote the extracted recommendation set as:ℛd=\{rd,1,rd,2,…,rd,md\},\\mathcal\{R\}\_\{d\}=\\\{r\_\{d,1\},r\_\{d,2\},\\dots,r\_\{d,m\_\{d\}\}\\\},wheremdm\_\{d\}is the number of recommendations extracted and mapped to diagnosis categorydd\.

Each recommendation setℛd\\mathcal\{R\}\_\{d\}is then converted by an LLM into executable diagnostic logic, yielding an initial disease\-specific skill:

gd\(0\)=LLMCompile​\(ℛd\)\.g\_\{d\}^\{\(0\)\}=\\mathrm\{LLM\}\_\{\\mathrm\{Compile\}\}\(\\mathcal\{R\}\_\{d\}\)\.\(6\)The initialized skill library is therefore defined as:

𝒢\(0\)=\{gd\(0\)∣d∈𝒟guide\},\\mathcal\{G\}^\{\(0\)\}=\\\{g\_\{d\}^\{\(0\)\}\\mid d\\in\\mathcal\{D\}\_\{\\mathrm\{guide\}\}\\\},\(7\)where𝒟guide\\mathcal\{D\}\_\{\\mathrm\{guide\}\}denotes the set of ICD\-10 diagnosis categories covered by the curated guidelines\.

Each skillgd\(0\)g\_\{d\}^\{\(0\)\}is implemented as an executable Python function\. It takes the relevant features of a patient casexxas input and returns a skill\-based evidence score:

sskill​\(x,d\)=gd\(0\)​\(x\)\.s\_\{\\mathrm\{skill\}\}\(x,d\)=g\_\{d\}^\{\(0\)\}\(x\)\.\(8\)The score is computed by matching the clinical evidence inxxagainst the executable diagnostic rules encoded ingd\(0\)g\_\{d\}^\{\(0\)\}\. Specifically, each skill assigns the evidence to one of four support levels:Confirmed,Strongly Suggestive,Compatible, andNot Supported\. Contradictory evidence is further used to downgrade the support level when applicable\. The final evidence score is normalized to the range\[0,1\]\[0,1\]\. Since each skill is executable, it can be directly invoked by an LLM agent to obtain a structured evidence score for diagnosisdd\. In this way,GuideSkill\-Zeroprovides a guideline\-grounded and interpretable mechanism for diagnosis scoring\.

### 3\.3Skill Evolution

AlthoughGuideSkill\-Zerois grounded in clinical guidelines, guidelines mainly capture general diagnostic principles and key decision points, while real\-world cases often contain heterogeneous presentations and atypical clinical patterns\. We therefore evolve the skill library using labeled patient cases\. Let the training set be𝒯=\{\(xj,dj\)\}j=1n\\mathcal\{T\}=\\\{\(x\_\{j\},d\_\{j\}\)\\\}\_\{j=1\}^\{n\}, wherexjx\_\{j\}is a patient case anddjd\_\{j\}is the ground\-truth diagnosis\.

For each case\(xj,dj\)\(x\_\{j\},d\_\{j\}\), the LLM generates a diagnostic rationalezjz\_\{j\}that summarizes the clinical evidence supportingdjd\_\{j\}\. Rationales associated with the same diagnosis are then aggregated and distilled into additional disease\-specific rules:

Δ​ℛd=LLMDistill​\(\{zj∣dj=d\}\)\.\\Delta\\mathcal\{R\}\_\{d\}=\\mathrm\{LLM\}\_\{\\mathrm\{Distill\}\}\\big\(\\\{z\_\{j\}\\mid d\_\{j\}=d\\\}\\big\)\.\(9\)
The newly distilled rules are used to update the skill library\. If diagnosisddalready has an existing skill, the rules are used to optimize that skill; otherwise, they are used to generate a new skill:

gd\(t\+1\)=\{Optimize​\(gd\(t\),Δ​ℛd\),if​d∈𝒟𝒢\(t\),Generate​\(Δ​ℛd\),otherwise\.g\_\{d\}^\{\(t\+1\)\}=\\begin\{cases\}\\mathrm\{Optimize\}\\big\(g\_\{d\}^\{\(t\)\},\\Delta\\mathcal\{R\}\_\{d\}\\big\),&\\text\{if \}d\\in\\mathcal\{D\}\_\{\\mathcal\{G\}^\{\(t\)\}\},\\\\ \\mathrm\{Generate\}\\big\(\\Delta\\mathcal\{R\}\_\{d\}\\big\),&\\text\{otherwise\}\.\\end\{cases\}\(10\)Here,𝒟𝒢\(t\)\\mathcal\{D\}\_\{\\mathcal\{G\}^\{\(t\)\}\}denotes the set of diagnoses currently covered by the skill library𝒢\(t\)\\mathcal\{G\}^\{\(t\)\}\. After applying this update across training diagnoses, we obtain the evolved skill library𝒢Evo\\mathcal\{G\}^\{\\mathrm\{Evo\}\}\.

This process improves both skill quality and skill coverage: existing skills become better aligned with real patient cases, while diagnoses missing from the initial guideline\-derived library can be newly added\.

### 3\.4Skill Execution

During inference, given an unseen patient casexx, the LLM first generates a ranked differential diagnosis set𝒞​\(x\)⊆𝒟\\mathcal\{C\}\(x\)\\subseteq\\mathcal\{D\}\. For each candidate diagnosisdi∈𝒞​\(x\)d\_\{i\}\\in\\mathcal\{C\}\(x\),GuideSkillretrieves the corresponding skill from the evolved skill library by matching either the disease name or the ICD\-10 code:

gdi←Retrieve​\(𝒢Evo,di\)\.g\_\{d\_\{i\}\}\\leftarrow\\mathrm\{Retrieve\}\(\\mathcal\{G\}^\{\\mathrm\{Evo\}\},d\_\{i\}\)\.\(11\)
Because different skills may require different clinical features, the LLM extracts the skill\-specific inputs needed by each retrieved skill:

ϕdi​\(x\)=LLMfeat​\(x,gdi\),\\phi\_\{d\_\{i\}\}\(x\)=\\mathrm\{LLM\}\_\{\\mathrm\{feat\}\}\(x,g\_\{d\_\{i\}\}\),\(12\)whereϕdi​\(x\)\\phi\_\{d\_\{i\}\}\(x\)denotes the subset of patient features required to executegdig\_\{d\_\{i\}\}\. The retrieved skill is then executed on these extracted features to produce a skill\-based evidence score:

sskill​\(x,di\)=gdi​\(ϕdi​\(x\)\)\.s\_\{\\mathrm\{skill\}\}\(x,d\_\{i\}\)=g\_\{d\_\{i\}\}\\big\(\\phi\_\{d\_\{i\}\}\(x\)\\big\)\.\(13\)
Meanwhile, the LLM\-generated differential diagnosis list provides a ranking over the candidate set\. We convert this rank into an LLM ranking score:

sLLM​\(x,di\)=1ranki\+1,s\_\{\\mathrm\{LLM\}\}\(x,d\_\{i\}\)=\\frac\{1\}\{\\mathrm\{rank\}\_\{i\}\+1\},\(14\)whereranki\\mathrm\{rank\}\_\{i\}is the zero\-based rank of candidatedid\_\{i\}\. The final fused score combines the LLM ranking score with the skill\-based evidence score:

sfuse​\(x,di\)=α​sLLM​\(x,di\)\+\(1−α\)​sskill​\(x,di\),s\_\{\\mathrm\{fuse\}\}\(x,d\_\{i\}\)=\\alpha s\_\{\\mathrm\{LLM\}\}\(x,d\_\{i\}\)\+\(1\-\\alpha\)s\_\{\\mathrm\{skill\}\}\(x,d\_\{i\}\),\(15\)whereα∈\[0,1\]\\alpha\\in\[0,1\]controls the relative contribution of the two signals\. By default, we setα=0\.5\\alpha=0\.5\. The final diagnosis is selected as:

d^=arg⁡maxdi∈𝒞​\(x\)⁡sfuse​\(x,di\)\.\\hat\{d\}=\\arg\\max\_\{d\_\{i\}\\in\\mathcal\{C\}\(x\)\}s\_\{\\mathrm\{fuse\}\}\(x,d\_\{i\}\)\.\(16\)
Compared with direct LLM inference, this process grounds the final decision in executable clinical rules while preserving the broad diagnostic capability of the LLM\.

![Refer to caption](https://arxiv.org/html/2607.26160v1/x2.png)Figure 2:Test\-split diagnosis distribution by ICD\-10 chapter across the four benchmarks\. Each horizontal bar shows the percentage of gold diagnoses belonging to each chapter\.Table 1:Benchmark statistics and skill coverage\.GuideSkill\-Evosubstantially improves test\-case skill coverage overGuideSkill\-Zero, increasing overall coverage by 43\.0 points\.Base ModelMethodMedCaseReasoningER\-ReasonMIMIC\-CDM\-FIMedThink\-BenchAverage![[Uncaptioned image]](https://arxiv.org/html/2607.26160v1/figures/icon/gpt.png)GPT\-5\.4Direct25\.7342\.2293\.6229\.0947\.67CoT25\.1743\.0689\.3636\.3648\.493\-shot ICL28\.9743\.8992\.5534\.5549\.99RAG23\.7143\.0693\.6232\.7348\.28LLM DDx25\.2840\.0090\.4334\.5547\.57LLM DDx \+ RAG25\.3939\.7290\.4327\.2745\.70GuideSkill\-Zero35\.0153\.0695\.7434\.5554\.59GuideSkill\-Evo39\.71\+37\.07%55\.00\+25\.31%96\.81\+3\.41%36\.36\+0\.00%56\.97\+13\.96%![[Uncaptioned image]](https://arxiv.org/html/2607.26160v1/figures/icon/claude.png)Claude\-Sonnet\-4\.6Direct23\.2741\.1192\.5534\.5547\.87CoT23\.9439\.1789\.3632\.7346\.303\-shot ICL25\.9539\.9492\.5534\.5548\.25RAG23\.8343\.0691\.4929\.0946\.87LLM DDx22\.0434\.4492\.5530\.9144\.99LLM DDx \+ RAG22\.1535\.5691\.4927\.2744\.12GuideSkill\-Zero32\.3347\.2294\.6825\.4549\.92GuideSkill\-Evo40\.27\+55\.18%50\.56\+17\.42%95\.74\+3\.45%40\.00\+15\.77%56\.64\+17\.39%![[Uncaptioned image]](https://arxiv.org/html/2607.26160v1/figures/icon/gemini.png)MedGemma27B\{\}\_\{\\text\{27B\}\}Direct19\.0232\.2291\.4925\.4542\.05CoT20\.0233\.8992\.5525\.4542\.983\-shot ICL20\.2530\.2891\.4927\.2742\.32RAG14\.5430\.8391\.4921\.8239\.67LLM DDx18\.5739\.1792\.5521\.8243\.03LLM DDx \+ RAG17\.9039\.4492\.5518\.1842\.02GuideSkill\-Zero23\.6044\.7292\.5527\.2748\.08GuideSkill\-Evo25\.95\+28\.15%48\.89\+23\.96%93\.62\+1\.16%30\.91\+13\.35%49\.84\+15\.83%![[Uncaptioned image]](https://arxiv.org/html/2607.26160v1/figures/icon/qwen.png)Qwen3\.59B\{\}\_\{\\text\{9B\}\}Direct19\.6939\.4492\.5521\.8243\.38CoT19\.5736\.1193\.6223\.6443\.243\-shot ICL20\.6939\.9492\.5523\.6444\.21RAG16\.5533\.3390\.4323\.6440\.99LLM DDx17\.6733\.3393\.6220\.0041\.16LLM DDx \+ RAG19\.1335\.2892\.5521\.8242\.20GuideSkill\-Zero23\.7141\.6794\.6825\.4546\.38GuideSkill\-Evo26\.17\+26\.49%46\.39\+16\.15%96\.81\+3\.41%34\.55\+46\.15%50\.98\+15\.31%

Table 2:Main results across four benchmarks and four base models, reported as accuracy \(%\)\. For each base model and column,boldindicates the best score, anditalicindicates the strongest baseline\.Greennumbers show the relative improvement ofGuideSkill\-Evoover the strongest baseline\.GuideSkill\-Evoachieves the best accuracy across all benchmarks and base models, with the largest gains over the strongest baselines andGuideSkill\-Zero\.

## 4Experiments

#### Data\.

We evaluateGuideSkill\-ZeroandGuideSkill\-Evoon four heterogeneous diagnostic\-reasoning benchmarks: MedCaseReasoning\(Wuet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib1)\), ER\-Reason\(Mehandruet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib5)\), MIMIC\-CDM\-FI\(Hageret al\.[2024](https://arxiv.org/html/2607.26160#bib.bib4)\), and MedThink\-Bench\(Zhouet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib35)\), spanning published case narratives, sequential emergency\-department reasoning, full\-information acute\-abdominal diagnosis, and multistep medical QA\. Training splits from the first three benchmarks are used for skill evolution, whereas MedThink\-Bench is reserved for external evaluation and excluded from evolution\. For cross\-benchmark comparison, we normalize reference diagnoses to three\-character ICD\-10 categories, such asK35for acute appendicitis, and evaluate category\-level rather than subtype\-level diagnosis\. We useClaude\-Sonnet\-4\.6for skill initialization and evolution\. The four test sets contain 1,403 cases across 448 ICD\-10 categories \(Table[1](https://arxiv.org/html/2607.26160#S3.T1)\), with their chapter\-level diagnosis distributions shown in Figure[2](https://arxiv.org/html/2607.26160#S3.F2)\.

#### Baselines

To assessGuideSkillagainst representative inference\-time alternatives under a common evaluation protocol, we compare six baselines: direct prompting, chain\-of\-thought prompting\(Weiet al\.[2022](https://arxiv.org/html/2607.26160#bib.bib41)\), 3\-shot in\-context learning\(Brownet al\.[2020](https://arxiv.org/html/2607.26160#bib.bib42)\), guideline RAG\(Lewiset al\.[2020](https://arxiv.org/html/2607.26160#bib.bib40)\), LLM\-generated differential diagnosis \(DDx\), and LLM\-generated DDx with guideline retrieval\. These baselines test whether performance gains can be explained by explicit reasoning, case demonstrations, access to guideline text, broader candidate generation, or their combination\. We evaluate every method usingGPT\-5\.4,Claude\-Sonnet\-4\.6,MedGemma\-27B, andQwen3\.5\-9B, covering proprietary and open\-weight, general\-purpose and medically specialized backbones\. We further compareGuideSkillwith representative parameter\-update and structured\-guideline alternatives onQwen3\.5\-9B: fine\-tuning with guidelines\(Stanieket al\.[2025](https://arxiv.org/html/2607.26160#bib.bib15)\), cases\(Wuet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib1)\), or both; RL with cases\(Chenet al\.[2024](https://arxiv.org/html/2607.26160#bib.bib10)\); guideline fine\-tuning followed by case\-based RL; and guidelines represented as decision trees\(Denget al\.[2026](https://arxiv.org/html/2607.26160#bib.bib17)\)\. These methods span the principal supervision sources, optimization strategies, and guideline representations relevant to our setting\. To ensure comparability, we implement them using the same backbone, data splits, evaluation protocol, and curated guideline corpus whenever applicable, rather than comparing with published results obtained on different tasks\. The cited methods therefore motivate matched baseline adaptations rather than exact reproductions\. For case\-based training, we reserve 20% of the training split for validation and use the remainder for optimization\.

#### Evaluation\.

We report accuracy on each benchmark and the macro\-average across the four benchmarks\. Gold diagnoses are normalized to three\-character WHO ICD\-10 categories during preprocessing, and cases without a reliable mapping to a single diagnostic category are excluded\. Because all methods are instructed to return an ICD\-10 category code, we use exact equality between the predicted and reference codes as the primary correctness criterion; malformed or code\-free outputs are counted as incorrect\. Appendix[H](https://arxiv.org/html/2607.26160#A8)provides the model and inference settings, baseline implementations, training configurations, and complete evaluation protocol\. Appendix[I](https://arxiv.org/html/2607.26160#A9)describes ICD\-10 normalization, data filtering, and benchmark splits\.

![Refer to caption](https://arxiv.org/html/2607.26160v1/x3.png)Figure 3:Accuracy ofGuideSkill\-ZeroandGuideSkill\-Evoon initially covered and newly added ICD\-10 categories\. Evolution improves all new\-skill settings while preserving or improving existing\-skill performance in 11 of 16 settings\.MethodMedCaseReasoningER\-ReasonMIMIC\-CDM\-FIMedThink\-BenchAverage![[Uncaptioned image]](https://arxiv.org/html/2607.26160v1/figures/icon/qwen.png)Qwen3\.59B\{\}\_\{\\text\{9B\}\}Direct Inference19\.6939\.4492\.5521\.8243\.38Fine\-tuning w/ Guidelines\(Stanieket al\.[2025](https://arxiv.org/html/2607.26160#bib.bib15)\)23\.9443\.3392\.5518\.1844\.50Fine\-tuning w/ Cases\(Wuet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib1)\)23\.2746\.9492\.5518\.1845\.24Fine\-tuning w/ Guidelines \+ Cases24\.5045\.2893\.6216\.3644\.94RL w/ Cases\(Chenet al\.[2024](https://arxiv.org/html/2607.26160#bib.bib10)\)25\.9542\.5092\.5516\.3644\.34Fine\-tuning w/ Guidelines \+ RL w/ Cases24\.9443\.0693\.6221\.8245\.86Guidelines as Decision Trees\(Denget al\.[2026](https://arxiv.org/html/2607.26160#bib.bib17)\)20\.5829\.1756\.3832\.7334\.72GuideSkill\-Zero \(Ours\)23\.7141\.6794\.6825\.4546\.38GuideSkill\-Evo \(Ours\)26\.1746\.3996\.8134\.5550\.98

Table 3:Training results on Qwen3\.5\-9B across four benchmarks, reported as accuracy \(%\)\. For each column,boldindicates the best score\.GuideSkill\-Evoachieves the best performance across all benchmarks and attains the highest average accuracy, consistently outperforming fine\-tuning and RL\-based training variants\.

## 5Results

#### Main Results

Table[2](https://arxiv.org/html/2607.26160#S3.T2)reports results across four backbone models\.GuideSkill\-Evoachieves the highest macro\-average accuracy for every backbone: 56\.97 withGPT\-5\.4, 56\.64 withClaude\-Sonnet\-4\.6, 49\.84 withMedGemma\-27B, and 50\.98 withQwen3\.5\-9B\. ForGPT\-5\.4andClaude\-Sonnet\-4\.6, these scores exceed the strongest non\-GuideSkillbaseline by 6\.98 and 8\.39 percentage points, respectively\. The gains therefore extend across proprietary, open\-weight, general\-purpose, and medically specialized backbones, indicating that the improvement is not tied to a particular LLM\.

Even without case\-based evolution,GuideSkill\-Zerooften ranks second within each backbone block and outperforms guideline RAG for every backbone\. This is notable because its guideline\-derived library covers only a subset of the evaluated diagnoses; candidates without a matching skill rely primarily on the LLM ranking\. Its competitive performance shows that guideline\-derived executable skills provide substantial diagnostic value before using benchmark training cases\. Rather than creating this benefit from scratch, evolution builds on it by expanding the library from 349 to 473 ICD\-10 categories and increasing gold\-label skill coverage from 56\.5% to 99\.5%\. The resultingGuideSkill\-Evoimproves overGuideSkill\-Zeroon MedCaseReasoning, ER\-Reason, and MedThink\-Bench\. On MedThink\-Bench, which is excluded from evolution, it outperformsGuideSkill\-Zeroacross all four backbones, suggesting that case\-derived updates can transfer to an unseen benchmark\.

The advantage over guideline RAG further highlights the value of operationalizing guideline knowledge\. Whereas RAG appends retrieved passages and relies on the LLM to interpret them,GuideSkillexecutes disease\-specific criteria to produce candidate\-level support scores\. To examine robustness, efficiency, and mechanism, additional analyses show thatα=0\.5\\alpha=0\.5is a competitive fusion setting and that accuracy largely saturates atK=5K=5\(Figures[4](https://arxiv.org/html/2607.26160#A1.F4)and[5](https://arxiv.org/html/2607.26160#A1.F5)\)\. Batched feature grounding inGuideSkill\-Effireduces estimated GPT\-5\.4 API cost by 72\.7% with a 0\.34\-point accuracy decrease on MedCaseReasoning \(Appendix[F](https://arxiv.org/html/2607.26160#A6); Table[6](https://arxiv.org/html/2607.26160#A2.T6)\)\. A case study further illustrates how candidate\-specific skill scores can correct an initially incorrect LLM ranking \(Appendix[K](https://arxiv.org/html/2607.26160#A11)\)\.

#### Skill Evolution

We analyze evolution from two complementary perspectives: skill coverage and downstream diagnostic utility\. Gold\-label skill coverage is the percentage of test cases whose reference ICD\-10 category has a corresponding executable skill\. As shown in Table[1](https://arxiv.org/html/2607.26160#S3.T1), evolution expands the library from 349 to 473 ICD\-10 categories and increases coverage from 56\.5% withGuideSkill\-Zeroto 99\.5% withGuideSkill\-Evo, filling nearly all gaps in the initial guideline\-derived library\. Figure[3](https://arxiv.org/html/2607.26160#S4.F3)examines whether this expansion improves newly covered diagnoses without degrading the original skills\. For newly covered categories,GuideSkill\-Evoimproves accuracy in all 12 available backbone–benchmark comparisons, with gains of 3\.3–22\.2 percentage points\. For initially covered categories, accuracy improves or remains unchanged in 11 of 16 comparisons; gains reach 7\.1 points, whereas declines are limited to 3\.6 points\. Evolution therefore substantially improves newly covered diagnoses while preserving or strengthening the initial skills in most settings\. To identify the remaining bottlenecks, we conduct an error analysis withClaude\-Sonnet\-4\.6\. Of the 749 errors, 456 \(60\.9%\) occur because the reference ICD\-10 category is absent from the candidate set and therefore cannot be recovered by downstream skill execution\. Candidate recall is thus the dominant remaining failure mode; Appendix[G](https://arxiv.org/html/2607.26160#A7)and Table[7](https://arxiv.org/html/2607.26160#A7.T7)provide the complete error breakdown\.

#### Comparison with Training\- and Guideline\-Based Baselines

Table[3](https://arxiv.org/html/2607.26160#S4.T3)comparesGuideSkillwith parameter\-update and structured\-guideline baselines using the sameQwen3\.5\-9Bbackbone\. GuideSkill\-Evo achieves the highest macro\-average accuracy of 50\.98, exceeding the strongest parameter\-update baseline by 5\.12 percentage points and Guidelines as Decision Trees by 16\.26 points\. It ranks first on three of four benchmarks; the only exception is ER\-Reason, where case fine\-tuning exceeds it by 0\.55 points\. Notably,GuideSkill\-Zeroalready achieves a macro\-average of 46\.38, surpassing the strongest parameter\-update baseline before case\-based skill evolution\. On MedThink\-Bench, which is excluded from skill evolution and case\-based parameter training,GuideSkill\-Evoachieves 34\.55, compared with 21\.82 for the strongest parameter\-update baseline and 32\.73 for Guidelines as Decision Trees\. This indicates stronger transfer to an unseen benchmark\. By storing diagnostic knowledge in external functions that directly execute disease\-specific criteria,GuideSkillimproves accuracy without updating the backbone while keeping its decision logic explicit\.

Table 4:Comparison between executable and textual skill representations using the sameGuideSkill\-Evocandidate sets\. Executable skills provide deterministic tier scoring after feature grounding, while textual skills rely on direct LLM interpretation of plain\-text rubrics\.
#### Comparison Between Executable and Textual Skills

Under identicalGuideSkill\-Evocandidate sets and fusion parameters, executable and textual skills achieve similar mean accuracy \(60\.94 versus 60\.60\), but only the executable variant shows no run\-to\-run accuracy variation over five evaluations, providing a more repeatable scoring interface \(Table[4](https://arxiv.org/html/2607.26160#S5.T4); Appendix[C](https://arxiv.org/html/2607.26160#A3)\)\.

#### Clinician Assessment of Skill Quality

One clinician evaluated ten disease skills, five guideline\-initialized and five case\-derived, covering 42 initial rules, 100 evolution cases, 120 final rules, and 51 held\-out cases\. For each skill, the clinician first assigned an expected support tier \(0–3\) and confidence score to held\-out cases without seeing the skill output, enabling comparison with the method\-assigned tiers\. For guideline\-initialized skills, the clinician then assessed whether each initial rule was consistent with its cited guideline passage and whether clinically important criteria were omitted or incorrectly encoded\. Next, the clinician rated whether each evolution case was suitable for skill evolution and whether it supported a new or revised rule\. Finally, after reviewing the rule changes and final skill, the clinician recommended accepting, revising, or rejecting the skill\. This staged protocol reduces anchoring from the skill output; Appendix[J](https://arxiv.org/html/2607.26160#A10)provides the complete questionnaire\.

As shown in Table[5](https://arxiv.org/html/2607.26160#S5.T5), 95\.2% of the initial rules were guideline\-consistent, and all five guideline\-initialized skills were judged to have no major clinical errors\. Among evolution cases, 85\.0% were rated as suitable for skill evolution, with 64\.7% supporting a new or revised rule\. On blinded held\-out cases, all correct diagnoses received a support tier of at least 2, and 86\.3% of skill outputs were within one tier of the clinician judgment\. The overall quadratic\-weightedκ\\kappawas 0\.60, indicating moderate clinician–skill agreement\. Finally, 9/10 final skills were accepted unchanged or with only minor revisions\. A detailed case review showed that the only final skill requiring major revision resulted from an evolution case that changed an originally correct rule into an inappropriate one by overgeneralizing a case\-specific pattern rather than capturing a generalizable diagnostic rule\. Overall, these results suggest thatGuideSkillproduces clinically validated and largely acceptable skills through both skill initialization and evolution, while highlighting the need for careful validation of case\-derived updates\.

Table 5:Clinician assessment of guideline\-initialized and case\-derived skills\. Tier\-agreement metrics compare the clinician’s blinded judgments with the corresponding skill outputs\.

## 6Conclusion

We introducedGuideSkill, an external reasoning layer that compiles guideline criteria into executable disease\-specific skills and evolves them from labeled cases without updating the backbone\. Across four benchmarks and four LLMs, guideline\-initialized skills outperform guideline RAG, while evolution improves all 16 direct\-inference comparisons and raises gold\-label skill coverage from 56\.5% to 99\.5%\. OnQwen3\.5\-9B,GuideSkill\-Evoalso exceeds the strongest matched parameter\-update baseline\. These results show that clinical knowledge can be maintained as a reusable, inspectable, and extensible execution layer across LLM backbones; broader clinician validation remains necessary before clinical use\.

## References

- Anthropic \(2026\)Claude sonnet 4\.6\.Note:Accessed July 21, 2026External Links:[Link](https://www.anthropic.com/news/claude-sonnet-4-6)Cited by:[Appendix H](https://arxiv.org/html/2607.26160#A8.SS0.SSS0.Px1.p1.1)\.
- T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell, S\. Agarwal, A\. Herbert\-Voss, G\. Krueger, T\. Henighan, R\. Child, A\. Ramesh, D\. M\. Ziegler, J\. Wu, C\. Winter, C\. Hesse, M\. Chen, E\. Sigler, M\. Litwin, S\. Gray, B\. Chess, J\. Clark, C\. Berner, S\. McCandlish, A\. Radford, I\. Sutskever, and D\. Amodei \(2020\)Language models are few\-shot learners\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 1877–1901\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2005.14165),[Link](https://arxiv.org/abs/2005.14165)Cited by:[Appendix H](https://arxiv.org/html/2607.26160#A8.SS0.SSS0.Px4.p1.2),[§4](https://arxiv.org/html/2607.26160#S4.SS0.SSS0.Px2.p1.1)\.
- L\. Cao, Q\. Chen, and Y\. Guo \(2026\)EHR\-rag: bridging long\-horizon structured electronic health records and large language models via enhanced retrieval\-augmented generation\.arXiv preprint arXiv:2601\.21340\.Cited by:[§1](https://arxiv.org/html/2607.26160#S1.p1.1)\.
- J\. Chen, Z\. Cai, K\. Ji, X\. Wang, W\. Liu, R\. Wang, J\. Hou, and B\. Wang \(2024\)Huatuogpt\-o1, towards medical complex reasoning with llms\.arXiv preprint arXiv:2412\.18925\.Cited by:[§4](https://arxiv.org/html/2607.26160#S4.SS0.SSS0.Px2.p1.1),[Table 3](https://arxiv.org/html/2607.26160#S4.T3.2.2.8.6.1)\.
- Z\. Chen, A\. H\. Cano, A\. Romanou, A\. Bonnet, K\. Matoba, F\. Salvi, M\. Pagliardini, S\. Fan, A\. Köpf, A\. Mohtashami,et al\.\(2023\)Meditron\-70b: scaling medical pretraining for large language models\.arXiv preprint arXiv:2311\.16079\.Cited by:[§B\.1](https://arxiv.org/html/2607.26160#A2.SS1.p1.4),[§1](https://arxiv.org/html/2607.26160#S1.p2.1)\.
- R\. Deng, G\. Martin, T\. Wang, G\. Zhang, Y\. Liu, C\. Weng, Y\. Wang, J\. F\. Rousseau, and Y\. Peng \(2026\)CPGPrompt: translating clinical guidelines into llm\-executable decision support\.arXiv preprint arXiv:2601\.03475\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2601.03475),[Link](https://arxiv.org/abs/2601.03475)Cited by:[Appendix H](https://arxiv.org/html/2607.26160#A8.SS0.SSS0.Px4.p2.1),[§1](https://arxiv.org/html/2607.26160#S1.p2.1),[§2](https://arxiv.org/html/2607.26160#S2.p1.1),[§4](https://arxiv.org/html/2607.26160#S4.SS0.SSS0.Px2.p1.1),[Table 3](https://arxiv.org/html/2607.26160#S4.T3.2.2.10.8.1)\.
- Google \(2026\)MedGemma 27b instruction\-tuned model card\.Note:Accessed July 21, 2026External Links:[Link](https://huggingface.co/google/medgemma-27b-it)Cited by:[Appendix H](https://arxiv.org/html/2607.26160#A8.SS0.SSS0.Px1.p1.1)\.
- P\. Hager, F\. Jungmann, R\. Holland, K\. Bhagat, I\. Hubrecht, M\. Knauer, J\. Vielhauer, M\. Makowski, R\. Braren, G\. Kaissis, and D\. Rueckert \(2024\)Evaluation and mitigation of the limitations of large language models in clinical decision\-making\.Nature medicine30\(9\),pp\. 2613–2622\.External Links:[Document](https://dx.doi.org/10.1038/s41591-024-03097-1),[Link](https://doi.org/10.1038/s41591-024-03097-1)Cited by:[3rd item](https://arxiv.org/html/2607.26160#A9.I1.i3.p1.1),[§1](https://arxiv.org/html/2607.26160#S1.p1.1),[§1](https://arxiv.org/html/2607.26160#S1.p4.1),[§4](https://arxiv.org/html/2607.26160#S4.SS0.SSS0.Px1.p1.1)\.
- E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen \(2022\)LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations,External Links:[Document](https://dx.doi.org/10.48550/arXiv.2106.09685),[Link](https://arxiv.org/abs/2106.09685)Cited by:[Appendix H](https://arxiv.org/html/2607.26160#A8.SS0.SSS0.Px5.p1.4)\.
- Institute of Medicine \(2011\)Clinical practice guidelines we can trust\.The National Academies Press,Washington, DC\.External Links:[Document](https://dx.doi.org/10.17226/13058),[Link](https://doi.org/10.17226/13058)Cited by:[§1](https://arxiv.org/html/2607.26160#S1.p1.1),[§2](https://arxiv.org/html/2607.26160#S2.p1.1)\.
- R\. Jin, M\. Xu, F\. Meng, G\. Wan, Q\. Cai, Y\. Jiang, J\. Han, Y\. Chen, W\. Lu, M\. Wang, Z\. Lan, Y\. Jiang, J\. Liu, D\. Wang, L\. Cong, and Z\. Zhang \(2025\)STELLA: towards a biomedical world model with self\-evolving multimodal agents\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.07.01.662467)Cited by:[§2](https://arxiv.org/html/2607.26160#S2.p2.1)\.
- W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica \(2023\)Efficient memory management for large language model serving with pagedattention\.InProceedings of the 29th Symposium on Operating Systems Principles,pp\. 611–626\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2309.06180),[Link](https://arxiv.org/abs/2309.06180)Cited by:[Appendix H](https://arxiv.org/html/2607.26160#A8.SS0.SSS0.Px1.p1.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. K"uttler, M\. Lewis, W\. Yih, T\. Rockt"aschel, S\. Riedel, and D\. Kiela \(2020\)Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 9459–9474\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2005.11401),[Link](https://arxiv.org/abs/2005.11401)Cited by:[Appendix H](https://arxiv.org/html/2607.26160#A8.SS0.SSS0.Px4.p1.2),[§4](https://arxiv.org/html/2607.26160#S4.SS0.SSS0.Px2.p1.1)\.
- B\. Li, T\. Meng, X\. Shi, J\. Zhai, and T\. Ruan \(2023\)Meddm: llm\-executable clinical guidance tree for clinical decision\-making\.arXiv preprint arXiv:2312\.02441\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2312.02441),[Link](https://arxiv.org/abs/2312.02441)Cited by:[§1](https://arxiv.org/html/2607.26160#S1.p2.1),[§2](https://arxiv.org/html/2607.26160#S2.p1.1)\.
- I\. Loshchilov and F\. Hutter \(2019\)Decoupled weight decay regularization\.InInternational Conference on Learning Representations,External Links:[Document](https://dx.doi.org/10.48550/arXiv.1711.05101),[Link](https://arxiv.org/abs/1711.05101)Cited by:[Appendix H](https://arxiv.org/html/2607.26160#A8.SS0.SSS0.Px5.p1.4)\.
- Z\. Ma, S\. Yang, Y\. Ji, X\. Wang, Y\. Wang, Y\. Hu, T\. Huang, and X\. Chu \(2026\)SkillClaw: let skills evolve collectively with agentic evolver\.arXiv preprint arXiv:2604\.08377\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2604.08377),[Link](https://arxiv.org/abs/2604.08377)Cited by:[§2](https://arxiv.org/html/2607.26160#S2.p2.1)\.
- D\. McDuff, M\. Schaekermann, T\. Tu, A\. Palepu, A\. Wang, J\. Garrison, K\. Singhal, Y\. Sharma, S\. Azizi, K\. Kulkarni,et al\.\(2025\)Towards accurate differential diagnosis with large language models\.Nature642,pp\. 451–457\.External Links:[Document](https://dx.doi.org/10.1038/s41586-025-08869-4),[Link](https://doi.org/10.1038/s41586-025-08869-4)Cited by:[§1](https://arxiv.org/html/2607.26160#S1.p1.1),[§1](https://arxiv.org/html/2607.26160#S1.p2.1)\.
- N\. Mehandru, N\. Golchini, N\. Garg, K\. T\. LeSaint, C\. J\. Nash, A\. Ramachandran, T\. Zack, L\. G\. McCoy, A\. Rodman, D\. Bamman, M\. Molina, and A\. Alaa \(2025\)Er\-reason: a benchmark dataset for llm\-based clinical reasoning in the emergency room\.arXiv preprint arXiv:2505\.22919\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2505.22919),[Link](https://arxiv.org/abs/2505.22919)Cited by:[2nd item](https://arxiv.org/html/2607.26160#A9.I1.i2.p1.1),[§1](https://arxiv.org/html/2607.26160#S1.p4.1),[§4](https://arxiv.org/html/2607.26160#S4.SS0.SSS0.Px1.p1.1)\.
- J\. Ni, Y\. Liu, X\. Liu, Y\. Sun, M\. Zhou, P\. Cheng, D\. Wang, E\. Zhao, X\. Jiang, and G\. Jiang \(2026\)Trace2Skill: distill trajectory\-local lessons into transferable agent skills\.arXiv preprint arXiv:2603\.25158\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2603.25158),[Link](https://arxiv.org/abs/2603.25158)Cited by:[§2](https://arxiv.org/html/2607.26160#S2.p2.1)\.
- D\. Oniani, X\. Wu, S\. Visweswaran, S\. Kapoor, S\. Kooragayalu, K\. Polanska, and Y\. Wang \(2024\)Enhancing large language models for clinical decision support by incorporating clinical practice guidelines\.In2024 IEEE 12th International Conference on Healthcare Informatics \(ICHI\),pp\. 694–702\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2401.11120),[Link](https://arxiv.org/abs/2401.11120)Cited by:[§1](https://arxiv.org/html/2607.26160#S1.p2.1),[§2](https://arxiv.org/html/2607.26160#S2.p1.1)\.
- OpenAI \(2026a\)GPT\-5\.4 model documentation\.Note:OpenAI API documentationAccessed July 21, 2026External Links:[Link](https://developers.openai.com/api/docs/models/gpt-5.4)Cited by:[Appendix H](https://arxiv.org/html/2607.26160#A8.SS0.SSS0.Px1.p1.1)\.
- OpenAI \(2026b\)Text\-embedding\-3\-small model documentation\.Note:OpenAI API documentationAccessed July 21, 2026External Links:[Link](https://developers.openai.com/api/docs/models/text-embedding-3-small)Cited by:[Appendix H](https://arxiv.org/html/2607.26160#A8.SS0.SSS0.Px4.p1.2)\.
- Qwen Team \(2026\)Qwen3\.5\-9B model card\.Note:Accessed July 21, 2026External Links:[Link](https://huggingface.co/Qwen/Qwen3.5-9B)Cited by:[Appendix H](https://arxiv.org/html/2607.26160#A8.SS0.SSS0.Px1.p1.1)\.
- M\. C\. Schubert, S\. Soyka, W\. Wick, and V\. Venkataramani \(2025\)Guideline\-incorporated large language model\-driven evaluation of medical records using medcheckllm\.JMIR Formative Research9,pp\. e53335\.External Links:[Document](https://dx.doi.org/10.2196/53335),[Link](https://formative.jmir.org/2025/1/e53335)Cited by:[§1](https://arxiv.org/html/2607.26160#S1.p1.1),[§1](https://arxiv.org/html/2607.26160#S1.p2.1),[§2](https://arxiv.org/html/2607.26160#S2.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo \(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2402.03300),[Link](https://arxiv.org/abs/2402.03300)Cited by:[Appendix H](https://arxiv.org/html/2607.26160#A8.SS0.SSS0.Px5.p1.4)\.
- S\. Shen, W\. Cheng, M\. Ma, A\. Turcan, M\. J\. Zhang, and J\. Ma \(2026a\)SKILLFOUNDRY: building self\-evolving agent skill libraries from heterogeneous scientific resources\.arXiv preprint arXiv:2604\.03964\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2604.03964),[Link](https://arxiv.org/abs/2604.03964)Cited by:[§2](https://arxiv.org/html/2607.26160#S2.p2.1)\.
- Y\. Shen, L\. Cao, S\. Du, Y\. Wang, J\. Zhou, H\. Peng, and Y\. Guo \(2026b\)MedGuideX: internalizing decision logic from executable guidelines into large language models for clinical reasoning\.arXiv preprint arXiv:2605\.26567\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2605.26567),[Link](https://arxiv.org/abs/2605.26567)Cited by:[§1](https://arxiv.org/html/2607.26160#S1.p1.1),[§1](https://arxiv.org/html/2607.26160#S1.p2.1),[§2](https://arxiv.org/html/2607.26160#S2.p1.1)\.
- M\. Staniek, A\. Sokolov, and S\. Riezler \(2025\)Training and evaluation of guideline\-based medical reasoning in llms\.arXiv preprint arXiv:2512\.03838\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2512.03838),[Link](https://arxiv.org/abs/2512.03838)Cited by:[§1](https://arxiv.org/html/2607.26160#S1.p2.1),[§2](https://arxiv.org/html/2607.26160#S2.p1.1),[§4](https://arxiv.org/html/2607.26160#S4.SS0.SSS0.Px2.p1.1),[Table 3](https://arxiv.org/html/2607.26160#S4.T3.2.2.5.3.1)\.
- A\. Tziakouri and F\. Menolascina \(2025\)Reinforcement learning for clinical reasoning: aligning llms with acr imaging appropriateness criteria\.arXiv preprint arXiv:2510\.05194\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2510.05194),[Link](https://arxiv.org/abs/2510.05194)Cited by:[§2](https://arxiv.org/html/2607.26160#S2.p1.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. Zhou \(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 24824–24837\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2201.11903),[Link](https://arxiv.org/abs/2201.11903)Cited by:[Appendix H](https://arxiv.org/html/2607.26160#A8.SS0.SSS0.Px4.p1.2),[§4](https://arxiv.org/html/2607.26160#S4.SS0.SSS0.Px2.p1.1)\.
- World Health Organization \(2019\)International statistical classification of diseases and related health problems, 10th revision\.Note:Version 2019External Links:[Link](https://icd.who.int/browse10/2019/en)Cited by:[§3\.1](https://arxiv.org/html/2607.26160#S3.SS1.p1.4)\.
- K\. Wu, E\. Wu, R\. Thapa, K\. Wei, A\. Zhang, A\. Suresh, J\. J\. Tao, M\. W\. Sun, A\. Lozano, and J\. Zou \(2025\)Medcasereasoning: evaluating and learning diagnostic reasoning from clinical case reports\.arXiv preprint arXiv:2505\.11733\.Cited by:[1st item](https://arxiv.org/html/2607.26160#A9.I1.i1.p1.1),[§1](https://arxiv.org/html/2607.26160#S1.p1.1),[§1](https://arxiv.org/html/2607.26160#S1.p2.1),[§1](https://arxiv.org/html/2607.26160#S1.p4.1),[§4](https://arxiv.org/html/2607.26160#S4.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2607.26160#S4.SS0.SSS0.Px2.p1.1),[Table 3](https://arxiv.org/html/2607.26160#S4.T3.2.2.6.4.1)\.
- G\. Xu, N\. Tang, X\. Li, T\. J\. Li, Z\. Zheng, W\. Jin, and Y\. Shi \(2026\)An empirical study of agent skills for healthcare: practice, gaps, and governance\.arXiv preprint arXiv:2605\.02709\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2605.02709),[Link](https://arxiv.org/abs/2605.02709)Cited by:[§2](https://arxiv.org/html/2607.26160#S2.p2.1)\.
- Z\. You, X\. Chen, A\. Vashishtha, S\. Du, G\. Erion\-Barner, H\. Mei, H\. Peng, and Y\. Guo \(2026\)Improving clinical diagnosis with counterfactual multi\-agent reasoning\.arXiv preprint arXiv:2603\.27820\.Cited by:[§1](https://arxiv.org/html/2607.26160#S1.p1.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2023\)Judging llm\-as\-a\-judge with mt\-bench and chatbot arena\.arXiv preprint arXiv:2306\.05685\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2306.05685),[Link](https://arxiv.org/abs/2306.05685)Cited by:[Appendix H](https://arxiv.org/html/2607.26160#A8.SS0.SSS0.Px6.p1.1)\.
- S\. Zhou, W\. Xie, J\. Li, Z\. Zhan, M\. Song, H\. Yang, C\. Espinoza, L\. Welton, X\. Mai, Y\. Jin, Z\. Xu, Y\. Chung, Y\. Xing, M\. Tsai, E\. Schaffer, Y\. Shi, N\. Liu, Z\. Liu, and R\. Zhang \(2025\)Automating expert\-level medical reasoning evaluation of large language models\.npj Digital Medicine\.External Links:[Document](https://dx.doi.org/10.1038/s41746-025-02208-7),[Link](https://doi.org/10.1038/s41746-025-02208-7)Cited by:[4th item](https://arxiv.org/html/2607.26160#A9.I1.i4.p1.1),[§1](https://arxiv.org/html/2607.26160#S1.p4.1),[§4](https://arxiv.org/html/2607.26160#S4.SS0.SSS0.Px1.p1.1)\.

## Appendix ALimitations

AlthoughGuideSkillsubstantially improves clinical reasoning accuracy by integrating executable clinical skills with LLM\-based reasoning, it still has several limitations\. First, the current skill library is built from a limited set of clinical guidelines\. Many high\-quality guidelines remain to be collected, curated, and incorporated, which may further improve the coverage and reliability of the skill library\. Second, for scalability, the current framework performs skill initialization and skill evolution in a fully automated manner using LLMs\. While this design makesGuideSkilleasy to scale, involving physicians in the verification process could further improve the quality and clinical validity of the generated skills, though at the cost of additional time and annotation effort\. Finally,GuideSkillis still a research prototype and is not intended for direct deployment in real clinical settings\. Our goal is to provide an initial step toward combining executable clinical knowledge with LLM\-based reasoning, and further clinical validation is required before practical use\.

![Refer to caption](https://arxiv.org/html/2607.26160v1/x4.png)Figure 4:Sensitivity analysis of the fusion weightα\\alphaon four clinical reasoning benchmarks\.α=1\\alpha=1corresponds to LLM\-only ranking, whileα=0\\alpha=0corresponds to skill\-only scoring\. The vertical dashed line marks the default settingα=0\.5\\alpha=0\.5\. Across datasets,GuideSkillremains relatively stable over a broad range ofα\\alpha, showing that fusing LLM ranking with executable skill scores provides robust diagnostic performance\.![Refer to caption](https://arxiv.org/html/2607.26160v1/x5.png)Figure 5:Sensitivity analysis of candidate set size\. We vary the number of candidate diagnosesKKgenerated by the LLM and report the average accuracy\. The dotted curve shows recall@KK, which serves as an upper\-bound ceiling\. AsKKincreases, the final accuracy ofGuideSkillquickly saturates, indicating that a moderate candidate set provides a practical balance between candidate coverage and selection difficulty\.
## Appendix BPreprocessing of Clinical Practice Guidelines

Our preprocessing pipeline serves two goals: \(A\) curating a corpus ofusableclinical practice guidelines from a large and heterogeneous source, and \(B\) compiling this free text into a library ofexecutabledisease\-level diagnostic skills\.

### B\.1Guideline Corpus Curation

We start from the publicly availableepfl\-llm/guidelinescorpus\(Chenet al\.[2023](https://arxiv.org/html/2607.26160#bib.bib8)\), which is commonly used as a pretraining corpus for medical LLMs\. The corpus contains37,97037\{,\}970clinical practice guideline \(CPG\) documents drawn from nine public sources\. However, because these documents were scraped from online sources, the raw collection is highly noisy: some documents have missing or empty body text, some are extremely short or excessively long, and many contain content that is not directly relevant to actionable clinical guidance\. In addition, the corpus varies substantially in length, ranging from55to over255,000255\{,\}000words, with a median of618618words\. We therefore apply a four\-stage cascade filter to curate a usable guideline corpus\.

#### Source filtering\.

We first retain only seven authoritative clinical guideline sources: NICE, PubMed, CMA, CDC, SPOR, WHO, and CCO\. We discard the crowd\-sourced and quality\-inconsistentwikidocsubset \(33,05833\{,\}058documents\), as well as the length\-extremeicrcsubset \(4949documents\)\. This reduces the corpus from37,97037\{,\}970to4,8634\{,\}863documents\.

#### Length filtering\.

We remove documents with missing body text or fewer than100100words, yielding4,6754\{,\}675documents\.

#### Usability filtering\.

Even authoritative sources contain tables of contents, reference lists, methodology sections, disclaimers, announcements, and abstracts without actionable guidance\. To remove such documents, we use an LLM\-as\-a\-judge,Claude\-Sonnet\-4\.6, to determine whether each document, truncated to its first12,00012\{,\}000characters, contains*actionable clinical guidance*, defined as concrete recommendations on diagnosis, treatment, screening, management, dosing, or eligibility\. Only documents judged usable are retained, leaving4,1454\{,\}145documents\.

#### Length\-outlier truncation\.

Finally, we remove the longest5%5\\%of documents using a p9595word\-count cutoff\. This prevents overlong documents from dominating downstream context and diluting relevant content\. The final curated corpus contains3,938clinical practice guidelines, with a median length of4,0304\{,\}030words, a mean length of5,0465\{,\}046words, and a range of101101–18,66218\{,\}662words\. By source, it comprises1,5851\{,\}585NICE,1,2361\{,\}236PubMed,377377CMA,374374CDC,184184SPOR,108108WHO, and7474CCO documents\.

### B\.2From Guidelines to Executable Diagnostic Skills

Free\-text guidelines cannot be executed directly\. We therefore compile the curated corpus into a disease\-indexed library of executable diagnostic skills through three steps\.

#### Recommendation extraction\.

UsingClaude\-Opus\-4\.8, we extractdisease\-diagnosisrecommendations from each guideline, namely rules of the form clinical findings or diagnostic criteria→\\rightarrowconfirm or rule out a specific diseaseXX\. Here, a recommendation refers to an actionable diagnostic rule that links observable patient evidence, such as symptoms, signs, laboratory results, imaging findings, or diagnostic criteria, to a disease\-level conclusion\. To ensure executability, we impose three strict constraints\. First, we retain only rules whose output is a disease\-level conclusion, explicitly excluding test\-appropriateness statements, such as whether to order a scan, even when they mention diagnosis or staging\. Second, each recommendation must map to a three\-character ICD\-10 category, such asK35; recommendations that do not map to a valid disease category, such as non\-disease findings codable only in the Z chapter, are discarded\. Third, each recommendation is annotated with a disease name, its ICD\-10 category, and anis\-usflag indicating whether the content is specific to United States clinical practice\. This step yields1,2001\{,\}200diagnostic recommendations spanning349349unique ICD\-10 disease categories, of which193193are marked as US\-specific\.

#### Merging by disease\.

Because the same disease is often covered by multiple guidelines, we merge all recommendations within each ICD\-10 category into a single self\-contained, complementary, and de\-duplicated diagnostic statement\. During merging, we reconcile US\-specific and non\-US content using theis\-usflag\. This produces one merged diagnostic statement for each ICD\-10 category, resulting in349349statements in total\.

#### Skill synthesis and indexing\.

Each merged statement is compiled into an executable Python diagnostic function, orskill, that takes structured features extracted from a patient case as input and returns a diagnostic confidence tier for the corresponding disease\. The result is an ICD\-10\-indexed library of349349executable diagnostic skills\. Overall, the pipeline distills37,97037\{,\}970heterogeneous documents into3,9383\{,\}938curated guidelines and further compiles them into349349disease\-organized, deterministically executable skills, which serve as the knowledge base for the retrieval and fusion stages of our method\.

Table 6:Accuracy, token usage, and estimated inference cost of different prompting and reasoning methods\.GuideSkillachieves the highest accuracy, whileGuideSkill\-Effimaintains nearly the same accuracy with substantially fewer output tokens and much lower inference cost\.

## Appendix CDetails of the Textual\-Skill Comparison

We compare executable and textual representations using identicalGuideSkill\-Evocandidate sets and the same fusion rule\. Executable skills apply Python functions to LLM\-grounded features to produce deterministic support tiers \(0–3\), whereas textual skills ask the LLM to assign tiers directly from equivalent plain\-text rubrics\. We evaluate 349 cases over five runs withClaude\-Sonnet\-4\.6at temperature 0 and report mean accuracy and run\-to\-run standard deviation\.

## Appendix DSensitivity Analysis of Score Fusion

Figure 2 analyzes the sensitivity ofGuideSkillto the fusion weightα\\alphaacross the four benchmarks\. Recall thatα\\alphacontrols the relative contribution of the LLM ranking score and the executable skill score:α=1\\alpha=1corresponds to relying only on the LLM ranking, whileα=0\\alpha=0corresponds to relying only on the skill score\. In our main experiments, we use the default settingα=0\.5\\alpha=0\.5\.

Overall, the fused scoring mechanism is robust across a broad range ofα\\alphavalues, but the best setting varies slightly by benchmark\. On MedCaseReasoning and ER\-Reason, performance peaks aroundα=0\.5\\alpha=0\.5, indicating that both the LLM’s ranking prior and the skill\-based evidence contribute useful information\. On MIMIC\-CDM\-FI, the best performance is obtained aroundα=0\.35\\alpha=0\.35, suggesting that the executable skills provide particularly strong diagnostic signal in this more structured setting\. On MedThink\-Bench, performance is highest nearα=0\\alpha=0, indicating that skill scores are more reliable than the LLM ranking for these compact but challenging reasoning cases\.

These results show that neither pure LLM ranking nor pure skill scoring is uniformly optimal across benchmarks\. Instead, combining the two sources of evidence yields stable performance, withα=0\.5\\alpha=0\.5serving as a simple default that performs competitively across datasets without benchmark\-specific tuning\.

## Appendix EEffect of Top\-K Candidate Diagnoses

Figure 3 analyzes the effect of the candidate set sizeKKon diagnostic accuracy based onClaude\-Sonnet\-4\.6\. In our main experiments, we setK=5K=5by default\. As expected, recall@KKconsistently increases as more candidate diagnoses are included, since a larger candidate set is more likely to contain the ground\-truth diagnosis\. However, the final accuracy ofGuideSkilldoes not always increase at the same rate, because the model must still select the correct diagnosis from a larger set of plausible candidates\. On MedCaseReasoning and ER\-Reason,GuideSkillquickly reaches a stable performance after a small number of candidates, indicating that most useful diagnostic evidence is already captured in the top\-ranked candidates\. On MIMIC\-CDM\-FI, performance improves with largerKKand then saturates, closely following the high recall ceiling\. On MedThink\-Bench, increasingKKbrings more noticeable gains, suggesting that challenging reasoning cases benefit from a broader candidate set\. Overall, this analysis shows thatK=5K=5provides a good trade\-off between candidate coverage and decision complexity, and supports its use as the default setting\.

## Appendix FEfficiency Analysis

We evaluate the efficiency ofGuideSkillagainst baseline methods to assess its potential for future deployment\. The originalGuideSkillframework performs skill\-level evidence extraction independently for each candidate skill, which enables accurate and fine\-grained reasoning but incurs relatively high inference cost\. To improve deployment efficiency,GuideSkill\-Effiperforms feature extraction in a unified pass, allowing shared case\-level evidence to be extracted once and reused across skills\. Specifically,GuideSkill\-Effiis an efficiency\-oriented variant that reduces the number of LLM calls per case from1\+N1\+Nto exactly two, whereNNis the number of matched skills\. InGuideSkill, inference consists of one differential\-diagnosis proposal followed by one feature\-extraction call for each matched skill\. InGuideSkill\-Effi, the first call proposes the top\-KKdifferential diagnosis as above, while the second call extracts the required features for*all*matched skills in a single pass\. The model is instructed to return only feature keys that are explicitly present in the case, with all other features defaulting to absent\. Each skill is then executed locally without further LLM calls, and fusion proceeds identically toGuideSkill\. Because feature extraction is a form\-filling task rather than a reasoning task,GuideSkill\-Effiruns both calls without reasoning\.

This design preserves the main benefit of skill\-guided reasoning while greatly reducing generation overhead\. We evaluate bothGuideSkilland its efficient variant against baseline methods on the MedCaseReasoning benchmark, where bothGuideSkillvariants use the evolved skill library\. As shown in Table 3,GuideSkill\-Effiachieves 39\.37% accuracy, only 0\.34 points lower thanGuideSkill, while reducing the estimated cost from 409\.01 to 111\.51 per 10k cases\.

Importantly,GuideSkill\-Effistill substantially outperforms all baseline methods in accuracy, while achieving the lowest inference cost among all compared methods\. This shows that the efficient design does not simply trade accuracy for lower cost, but provides a better accuracy\-cost balance\. This reduction is particularly meaningful because, in many commercial LLM APIs, output tokens are priced higher than input tokens\. GPT\-5\.4 follows this common pricing pattern: the official API price is $2\.50 per 1M input tokens and $15\.00 per 1M output tokens, so output tokens are 6×\\timesmore expensive than input tokens111https://developers\.openai\.com/api/docs/models/gpt\-5\.4\. As a result, inference cost is often dominated by output length\.GuideSkill\-Effidirectly addresses this bottleneck by transferring much of the reasoning process from free\-form LLM generation to structured executable skills, yielding much shorter outputs with little accuracy degradation\.

## Appendix GError Analysis ofGuideSkill\-Evo

Table 7:Error analysis ofGuideSkill\-EvowithClaude\-Sonnet\-4\.6across all test cases\. Stage percentages indicate each error stage’s share among all analyzed errors; clinical\-type percentages are computed within each stage\.AlthoughGuideSkill\-Evoachieves the best overall performance, it still leaves substantial room for improvement\. We conduct an error analysis on all test cases usingClaude\-Sonnet\-4\.6as the backbone and categorize the remaining errors into three stages\. The analysis is judge and cateogirze also byClaude\-Sonnet\-4\.6Recall\-misserrors occur when the gold diagnosis is not included in the LLM\-generated candidate set\.Skill\-underscoringerrors occur when the gold diagnosis is recalled but its executable skill assigns an insufficient score\.Lost\-to\-distractorerrors occur when the gold diagnosis is recalled and scored, but another plausible candidate receives a higher fused score\.

As shown in Table 6, the largest source of error is recall failure: in 456 cases, the gold ICD\-10 category is not included in the candidate set, making it impossible for skill execution to recover the correct answer\. Most recall\-miss errors are due to coding granularity, where the LLM proposes a clinically related diagnosis but not the exact ICD\-10 category required by evaluation\. Among cases where the gold diagnosis is recalled, errors often arise from fine\-grained clinical distinctions\. Skill\-underscoring errors are dominated by adjacent same\-system diagnoses and cross\-system mimics, suggesting that some skills still underweight discriminative evidence\. Lost\-to\-distractor errors show a similar pattern: the correct diagnosis is present, but a nearby diagnosis, causal downstream condition, or cross\-system mimic receives a stronger fused score\. These results suggest that future improvements should target both candidate recall at the ICD\-10 category level and finer\-grained skill calibration among clinically similar diagnoses\.

## Appendix HExperiment Details

#### Models\.

We evaluate all methods using four backbone LLMs spanning both proprietary and open\-weight models\. The proprietary models are accessed through Microsoft Azure222https://azure\.microsoft\.com/en\-us: Claude\-Sonnet\-4\.6 \(claude\-sonnet\-4\-6\)\(Anthropic[2026](https://arxiv.org/html/2607.26160#bib.bib49)\)and GPT\-5\.4 \(gpt\-5\.4\)\(OpenAI[2026a](https://arxiv.org/html/2607.26160#bib.bib48)\)\. The open\-weight models are served locally with vLLM\(Kwonet al\.[2023](https://arxiv.org/html/2607.26160#bib.bib46)\): MedGemma\-27B \(google/medgemma\-27b\-it\)\(Google[2026](https://arxiv.org/html/2607.26160#bib.bib50)\)and Qwen3\.5\-9B \(Qwen/Qwen3\.5\-9B\)\(Qwen Team[2026](https://arxiv.org/html/2607.26160#bib.bib51)\)\. We use Claude\-Sonnet\-4\.6 for skill initialization and skill evolution inGuideSkill\. For experiments involving non\-public clinical data from PhysioNet, we follow the PhysioNet responsible\-use guidance for MIMIC data with LLMs333https://physionet\.org/news/post/llm\-responsible\-use/\. Specifically, restricted clinical data are used with proprietary models only through an institutionally approved Azure deployment that ensures zero data retention, no training on submitted data, and no human review of prompts or outputs\. The locally served open\-weight models provide a fully controlled deployment path\.

#### LLM Inference\.

Proprietary models are queried through the Azure API\. For Claude backbones, we set temperature to0and do not enable extended thinking\. GPT\-5\.4 is queried with the API default settings, including the default temperature of11and the default reasoning configuration\. Open\-weight models are served through vLLM with a32,76832\{,\}768\-token context window\. For Qwen3\.5\-9B, we set temperature to0and explicitly disable thinking mode by settingenable\_thinking=False, so that the final answer is returned directly rather than embedded in a long reasoning trace\. For MedGemma\-27B, we use temperature0; since it is not a reasoning model, no thinking mode is used\. Unless otherwise noted, generation is capped at4,0964\{,\}096tokens\.

#### Skill Evolution\.

We evolve the skill library using a merged training set from MedCaseReasoning, ER\-Reason, and MIMIC\-CDM\-FI, with no overlap with the test splits\. Each case is labeled with its three\-character ICD\-10 category\. The evolution pipeline mirrors skill initialization and also usesclaude\-sonnet\-4\-6\. First, in*rationale generation*, the model is given each training case and its gold diagnosis, and is asked to extract the key clinical variables and explain why they support that diagnosis\. Second, in*case\-to\-recommendation*, cases are grouped by ICD\-10 label and distilled into a single recommendation that captures only criteria recurring across cases, while discarding case\-specific incidentals\. When a matching guideline recommendation already exists, it is used as the backbone and is never weakened\. We process cases in batches of1010using a map\-reduce strategy: the model first drafts recommendations for each batch and then iteratively refines them, preventing prompt overflow and reducing specificity loss\. Third, in*recommendation\-to\-skill*, the resulting recommendations are compiled using the same generator and tier contract as in skill initialization\. Skill evolution yields473473case\-derived skills\. After merging them with the initialized skill library, we obtain the evolved skill library covering473473ICD\-10 categories, including206206categories shared with the initialized library and267267newly added categories from training cases\.

#### Baselines\.

We compareGuideSkillwith both prompting\-based and training\-based baselines, using the same backbone models whenever applicable\. For prompting\-based baselines, we include: \(i\)Direct, which directly prompts the model to produce the final diagnosis; \(ii\)CoT, which performs chain\-of\-thought diagnostic reasoning before producing the final answer\(Weiet al\.[2022](https://arxiv.org/html/2607.26160#bib.bib41)\); \(iii\)3\-shot ICL, which uses three fixed in\-domain demonstrations from each dataset’s training split\(Brownet al\.[2020](https://arxiv.org/html/2607.26160#bib.bib42)\)\. Since MedThink\-Bench does not provide a training split, we use demonstrations from MedCaseReasoning; \(iv\)RAG, which retrieves from curated guideline recommendations and prepends the top\-55passages retrieved bytext\-embedding\-3\-small\(OpenAI[2026b](https://arxiv.org/html/2607.26160#bib.bib52)\)as context before answering\(Lewiset al\.[2020](https://arxiv.org/html/2607.26160#bib.bib40)\); \(v\)LLM DDx, a two\-pass differential\-diagnosis baseline that first proposes the top\-55candidate diagnoses and then selects one final answer without using external knowledge; and \(vi\)LLM DDx \+ RAG, which extends the two\-pass differential\-diagnosis baseline by conditioning the final selection on the same retrieved guideline context used in RAG\.

For training\-based baselines, we conduct comparisons onQwen\-3\.5and include: \(vii\)Fine\-tuning w/ Guidelines, which fine\-tunes the model on all curated guideline recommendations; \(viii\)Fine\-tuning w/ Cases, which fine\-tunes the model on training cases with gold diagnoses; \(ix\)Fine\-tuning w/ Guidelines \+ Cases, which fine\-tunes on the union of guideline\-derived supervision and case\-level supervision; \(x\)RL w/ Cases, which applies reinforcement learning on training cases using diagnosis correctness as the reward signal; and \(xi\)Fine\-tuning w/ Guidelines \+ RL w/ Cases, which first fine\-tunes the model with guideline supervision and then further optimizes it with case\-level reinforcement learning\. All training baselines are implemented with verl444https://github\.com/verl\-project/verlusing the default training configuration\. We also include \(xii\)Guidelines as Decision Trees, which converts guideline recommendations into decision\-tree\-style reasoning structures and uses them as explicit diagnostic guidance\. We use the official implementation released by the original paper\(Denget al\.[2026](https://arxiv.org/html/2607.26160#bib.bib17)\)\.

#### Details of Training Baselines\.

For all trainable variants, we apply LoRA\(Huet al\.[2022](https://arxiv.org/html/2607.26160#bib.bib43)\)to all linear layers with rank 16 andα=32\\alpha=32\. Training uses bfloat16 precision, gradient checkpointing, AdamW optimization\(Loshchilov and Hutter[2019](https://arxiv.org/html/2607.26160#bib.bib44)\), a learning rate of1×10−51\\times 10^\{\-5\}, a constant learning\-rate schedule with 10 warmup steps, a global batch size of 512, a maximum sequence length of 8,192 tokens, and three training epochs\. For case\-based learning, we randomly split the original training set into training and validation subsets using an 8:2 ratio\. Guideline SFT is performed on the clinical guideline corpus, whereas case SFT is conducted on standardized diagnosis cases using answer\-only supervision\. TheGuidelines \+ Casessetting performs these two stages sequentially\. For RL, we adopt GRPO\(Shaoet al\.[2024](https://arxiv.org/html/2607.26160#bib.bib45)\)with a learning rate of5×10−65\\times 10^\{\-6\}, two training epochs, a batch size of 32, and 24 rollouts per prompt\. The maximum prompt and response lengths are set to 8,192 and 512 tokens, respectively\. We use one policy\-update epoch, a KL coefficient of 0\.005, an entropy coefficient of 0, a rollout temperature of 1\.0, and top\-ppof 1\.0\. Checkpoints are saved after every SFT epoch and every 10 RL updates, and the best checkpoint is selected based on validation performance\. All experiments are conducted on a single server equipped with eight NVIDIA RTX 5090 GPUs\.

#### Evaluation Protocol\.

We evaluate predictions at the three\-character ICD\-10 category level: a prediction is counted as correct if it maps to the same ICD\-10 category as the gold diagnosis\. Semantic matching between the predicted diagnosis and the gold label is performed using an LLM\-as\-a\-judge,claude\-haiku\-4\-5, with temperature set to0\. The judge treats synonyms, abbreviations, subtype\-level differences, and clinically equivalent paraphrases as matches when they fall within the same ICD\-10 category\. LLM judges can exhibit systematic biases and reasoning limitations\(Zhenget al\.[2023](https://arxiv.org/html/2607.26160#bib.bib47)\); we therefore identify judge\-based semantic matching as a limitation of the evaluation\. The full LLM\-as\-a\-judge prompt is shown below\.

LLM Judge Prompt for ICD\-10 Category MatchingDecide whether the predicted diagnosis and the ground\-truth diagnosis refer to the same disease at the ICD\-10 3\-character category level \(e\.g\., "K35"\)\.Judge at the category granularity: they match if they fall under the same ICD\-10 3\-character category, even if the subtype or wording differs\.Predicted diagnosis: \{pred\}Ground\-truth diagnosis: \{gold\}Please answer directly with "yes" or "no"\.

## Appendix IBenchmark Details and Examples

The four benchmarks originate from different data sources and represent case reports, emergency\-department records, structured clinical decision\-making cases, and medical QA vignettes\. Training splits from MedCaseReasoning, ER\-Reason, and MIMIC\-CDM\-FI are used for skill evolution and parameter\-update baselines; the corresponding held\-out test splits and the MedThink\-Bench test set are used for evaluation\. The benchmark data are separate from the guideline corpus used to initializeGuideSkill\-Zero\.

- •MedCaseReasoning\(Wuet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib1)\): A long\-form open diagnostic reasoning benchmark based on open\-access clinical case reports from the New England Journal of Medicine Clinicopathological Conferences \(NEJM CPC\)\. It evaluates case\-based differential diagnosis over broad and long\-tailed clinical conditions\. Example: MedCaseReasoningCase: A 30\-year\-old man presented with several months of pain, tenderness, and swelling on the left side of his palate\. T2\-weighted MRI showed a hyperintense lesion, and biopsy suggested a soft\-tissue tumor\. Given the case, what is the final diagnosis? \[case report excerpt; approximately 890 words\]Diagnosis: Other disorders of nerve roots and plexuses\. ICD\-10: G54\.
- •ER\-Reason\(Mehandruet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib5)\): An emergency\-department diagnosis prediction benchmark derived from real emergency\-room patient records\. It evaluates whether a model can infer the final diagnosis from noisy, time\-sensitive clinical presentations in the emergency setting\. Example: ER\-ReasonCase: Age: 26; Sex: Female; Chief Complaint: hematuria\. The record includes a full emergency\-department note with pregnancy and delivery history, normal spontaneous vaginal delivery, second\-degree perineal laceration repair, hematuria with dysuria, lower abdominal pain, and sexual history\. \[long EHR excerpt; approximately 15,400 words\]Diagnosis: Tubulo\-interstitial nephritis, not specified as acute or chronic\. ICD\-10: N12\.
- •MIMIC\-CDM\-FI\(Hageret al\.[2024](https://arxiv.org/html/2607.26160#bib.bib4)\): A full\-information open clinical decision\-making benchmark derived from MIMIC\-IV, which is based on electronic health records from Beth Israel Deaconess Medical Center\. It evaluates full\-information clinical decision making\. Example: MIMIC\-CDM\-FICase: Patient history describes 10 hours of abdominal pain that began near the umbilicus and migrated to the right lower quadrant, with one episode of diarrhea and no nausea, vomiting, or fever\. Physical examination shows right\-lower\-quadrant tenderness, positive obturator sign, and negative Rovsing sign\. Laboratory tests include urea nitrogen, sodium, potassium, lipase, creatinine, and other values\. \[structured history, exam, and laboratory excerpt; approximately 4,000 words\]Diagnosis: Acute appendicitis\. ICD\-10: K35\.
- •MedThink\-Bench\(Zhouet al\.[2025](https://arxiv.org/html/2607.26160#bib.bib35)\): An expert\-curated medical reasoning benchmark constructed from ten publicly available medical QA datasets\. The benchmark filters for complex questions requiring multi\-step reasoning across ten medical domains\. In our setting, we use it as a compact but challenging open\-ended diagnosis\-oriented reasoning benchmark\. Example: MedThink\-BenchCase: A 23\-year\-old female weightlifter presents with neck and right shoulder pain\. Osteopathic structural examination reveals restricted motion of Sibson’s fascia, and the clavicles appear asymmetric\. What is the most likely diagnosis? \[medical QA excerpt; approximately 780 words\]Diagnosis: Other acquired deformities of musculoskeletal system\. ICD\-10: M95\.

#### Preprocessing and Data Split\.

To normalize answer labels across benchmarks, we first map all gold diagnoses to ICD\-10 categories\. Specifically, we useClaude\-Sonnet\-4\.6to convert each dataset’s original answer label into an ICD\-10 code\. We discard examples for which no reliable ICD\-10 mapping can be obtained\. We also remove examples whose gold answer does not correspond to a single disease diagnosis, since our evaluation requires one normalized diagnostic target per case\.

After preprocessing, we randomly split MedCaseReasoning, ER\-Reason, and MIMIC\-CDM\-FI into training and test sets\. MedThink\-Bench does not provide a training split in our setting, so we use it only for evaluation\. The final processed data contain 11,598 training and 894 test cases for MedCaseReasoning, 1,235 training and 360 test cases for ER\-Reason, 219 training and 94 test cases for MIMIC\-CDM\-FI, and 55 test cases for MedThink\-Bench\. In total, our experiments use 13,052 training cases and 1,403 test cases after ICD\-10 normalization and filtering\.

## Appendix JClinician Assessment Questionnaire

This appendix gives the complete instrument used for the clinician assessment reported in Table[5](https://arxiv.org/html/2607.26160#S5.T5), together with the order in which material was presented\. The study was delivered as a single\-page web application; each packet covers one disease skill, and the clinician worked through the screens in a fixed order without being able to see later material in advance\.

### J\.1Presentation Order

Each packet is presented as four screens\. Critically, the*held\-out screen comes first*: the clinician assigns an expected support tier to unseen cases before any part of the skill — guideline passages, rule tables, or code — is revealed\. Presenting the skill first would let its output anchor the expected tiers and inflate agreement\.

1. 1\.Held\-out cases\(Q5\)\. Blinded tier assignment on cases never used to build the skill\. The skill’s own output stays hidden throughout\.
2. 2\.Guideline and initial skill\(Q1, Q2\)\. Numbered guideline passagesG1–Gnn, a human\-readable rule tableR1–Rnnlinking each rule to its supporting passage, and the initial Python function in an expandable panel\.
3. 3\.Evolution cases\(Q3\)\. Ten cases per skill, unlabeled and in fixed order, with the final skill still hidden\.
4. 4\.Initial\-to\-final update\(Q4\)\. The change log with per\-change provenance, the final rule table, and the final Python function; the disposition is recorded last\.

For*case\-derived*skills there is no guideline\-initialized predecessor, so Screen 2 and questions Q1–Q2 are omitted \(three screens, Q3–Q5\)\. These packets are explicitly labeled “case\-derived skill; no guideline\-derived initialization\.” In both tracks, “cannot assess / outside expertise” is available on every question and is excluded from the corresponding denominator rather than treated as a negative rating\.

### J\.2Questions

#### Q1 — Guideline support \(per rule; guideline\-initialized skills only\)\.

*For each rule in the initial skill, how well is it supported by the provided guideline passages?*

- •Fully supported
- •Mostly supported \(minor interpretation\)
- •Partially supported \(substantial interpretation\)
- •Unsupported or contradictory
- •Cannot assess

Table[5](https://arxiv.org/html/2607.26160#S5.T5)reports two thresholds on this scale: rules rated*fully or mostly supported*, and rules*not*rated unsupported or contradictory\.

#### Q2 — Errors or omissions in the initial skill \(per skill\)\.

*Based only on the provided guideline passages, does the initial skill omit or incorrectly encode any clinically important criterion, threshold, exclusion, exception, or logical relationship?*

- •No clinically important problem
- •Minor problem \(unlikely to change the support tier\)
- •Major problem \(could change the support tier\)
- •Potentially dangerous / seriously misleading
- •Cannot assess

Q1 and Q2 are locked before the clinician proceeds to the evolution cases\.

#### Q3 — Contribution of an evolution case \(per case\)\.

*If a rule for this skill were written from this case, what would it do to future patients?*

- AAdd or change a rule — shows a criterion the skill needs and might otherwise miss
- BKeep the rules as they are — textbook presentation; confirms what a skill would already check
- CWould apply to almost no one else — a rule from this case would rarely fire again; harmless but useless
- DWould fire on the wrong patients — a rule from this case would misjudge future patients \(scores a mimic as this disease, drops a needed requirement, or relies on a non\-specific feature\)
- ECannot assess

The deciding test between C and D is stated in the instrument:*would a rule taken from this case ever fire on a patient who does not have this disease? No→\\rightarrowC; yes→\\rightarrowD\.*An optional free\-text field records the finding or omission that drove the rating\. Cases rated A, B, or C are counted as generalizable in Table[5](https://arxiv.org/html/2607.26160#S5.T5); D marks a case that should not influence the skill\.

#### Q4 — Final update and disposition \(per skill\)\.

*After reviewing the guideline, the cases, and the initial\-to\-final diff, what is your recommendation for the final skill?*

- •Accept unchanged
- •Accept with minor edits
- •Major revision required
- •Reject
- •Cannot assess

If the recommendation is not “accept unchanged”, the clinician selects all applicable concerns from: insufficient case support; case\- or dataset\-specific pattern; conflicts with the guideline; weakens a guideline\-supported requirement; omits an important criterion; treats an unreported feature as absent; treats an unexcluded mimic as excluded; incorrect threshold, unit, negation, or AND/OR logic; inappropriate support tier; other\. A free\-text field records the single most important required change\.

#### Q5 — Expected support tier \(per held\-out case\)\.

*Based on the available evidence, what diagnostic\-support tier should a skill for this disease assign to this case?*

- •3 — Confirmed / highest support
- •2 — Strongly suggestive
- •1 — Compatible
- •0 — Not supported
- •Insufficient information to assign a tier
- •Cannot assess

A confidence rating \(low / moderate / high\) accompanies each tier\. These are the same four tiers the skills themselves emit, so the clinician’s blinded judgment and the skill’s executed tier are directly comparable; agreement is computed only after all held\-out cases for a packet are submitted\.

### J\.3Held\-Out Case Construction

Held\-out cases are drawn from the MedCaseReasoning test split and are disjoint from every case used to build the skill\. Each packet mixes two kinds, and the kind is*not*shown to the clinician:

- •Correct diagnoses\(23 cases\): the reviewed category is the gold diagnosis\.
- •Look\-alikes\(28 cases\): the gold diagnosis is a different category, but the reviewed category appeared in the model’s proposed differential — an operational, reproducible definition of clinical similarity\.

Because both kinds are presented identically, Q5 measures whether the skill’s tier tracks clinical judgment on cases that do and do not warrant support, rather than only on positives\.

## Appendix KCase Study

We present a representative MedCaseReasoning example as a case study ofGuideSkill\. In this example,GuideSkillcorrects the base LLM’s plausible but incorrect top\-ranked diagnosis\. The patient is a 60\-year\-old woman with dyspepsia, unintentional weight loss, postprandial vomiting, gastric ulcers on endoscopy, and biopsy showing large confluent non\-caseating epithelioid granulomas\. The base LLM initially ranks Crohn’s disease as the most likely diagnosis, likely because gastrointestinal ulcers and granulomas are common cues for Crohn’s disease\. However, after executing candidate\-specific skills,GuideSkillassigns different diagnostic strengths to the same evidence\. The Crohn’s disease skill treats granulomas as a specific but insufficient feature and stops at tier 2, while the sarcoidosis skill treats non\-caseating granulomas as confirmatory histologic evidence and assigns tier 3\. The fusion step therefore overturns the LLM ranking and selects sarcoidosis, matching the gold diagnosis\.

The key mechanism is that the two executed skills assign different evidential status to the same grounded finding\. For sarcoidosis, non\-caseating granulomas with unsupported mimics enter a direct tier\-3 branch\. For Crohn’s disease, granulomas alone are only a specific tier\-2 feature unless additional Crohn’s\-specific criteria or mimic\-exclusion requirements are satisfied\. Thus, the same clinical evidence is sufficient to confirm sarcoidosis but only suggestive for Crohn’s disease, making the tier\-based fusion interpretable\.

Table 8:Case study on MedCaseReasoning\. The table includes the patient case, ground\-truth diagnosis, the LLM’s top\-5 differential diagnosis, and the executable skill trace used byGuideSkill\.`Executed Skill: Sarcoidosis \(D86\)``Executed Skill: Crohn’s Disease \(K50\)`
## Appendix LPrompt Design

The three stages ofGuideSkillinvolve multiple LLM calls, with different prompts designed for preprocessing, skill construction, and inference\-time reasoning\.

In theSkill Initializationstage, four prompts are used:ICD\-10 Label Normalizationmaps free\-text diagnoses to ICD\-10 categories;Guideline to Recommendationextracts atomic diagnostic recommendations from clinical guidelines;Recommendation Mergeconsolidates recommendations that map to the same ICD\-10 category; andRecommendation to Executable Skillcompiles each merged recommendation into an executable diagnostic skill\.

In theSkill Evolutionstage, three prompts are used:Rationale Generationderives diagnosis\-supporting rationales from training cases;Case\-to\-Recommendation Distillationdistills recurring diagnostic criteria across cases into disease\-level recommendations; andCase\-to\-Recommendation Refinementupdates existing recommendations with new batches of case evidence\.

In theSkill Executionstage, three prompts are used:Differential Diagnosis Proposalproposes a ranked ICD\-10 differential diagnosis;Skill Feature Groundingmaps a patient case into the input features required by a selected skill; andEfficient Skill Feature Groundinggrounds features for multiple candidate skills in a single pass, which is used by the efficient variant ofGuideSkill\.

The full prompt templates are shown below\. Placeholders to be replaced at runtime are enclosed in curly braces \(\{ \}\)\.

`Skill Initialization: ICD\-10 Label Normalization``Skill Initialization: Guideline to Recommendation``Skill Initialization: Recommendation Merge``Skill Initialization: Recommendation to Executable Skill``Skill Evolution: Rationale Generation``Skill Evolution: Case\-to\-Recommendation Distillation``Skill Evolution: Case\-to\-Recommendation Refinement``Skill Execution: Differential Diagnosis Proposal``Skill Execution: Skill Feature Grounding``Skill Execution: Efficient Skill Feature Grounding`