SafeLLM: Extraction as a Hallucination-Resistant Alternative to Rewriting in Safety-Critical Settings

# SafeLLM: Extraction as a Hallucination-Resistant Alternative to Rewriting in Safety-Critical Settings
Source: [https://arxiv.org/html/2606.12897](https://arxiv.org/html/2606.12897)
Julia IveInstitute of Health Informatics, University College London, London, UKThese authors contributed equally: Julia Ive, Felix Jozsajulia\.ive@ucl\.ac\.ukFelix JozsaInstitute of Health Informatics, University College London, London, UKNational Hospital for Neurology and Neurosurgery, Queen Square, London, UKThese authors contributed equally: Julia Ive, Felix JozsaNabeel SheikhSomerset NHS Foundation Trust, UKEmma CattellSomerset NHS Foundation Trust, UKNick JacksonKing’s College Hospital, Denmark Hill, London, UKPaulina BondaronekInstitute of Health Informatics, University College London, London, UKCiaran Scott HillNational Hospital for Neurology and Neurosurgery, Queen Square, London, UKRichard DobsonInstitute of Health Informatics, University College London, London, UKKing’s College London, London, UK

###### Abstract

Background:Large language models \(LLMs\) are increasingly used to access organisational documentation, including standard operating procedures \(SOPs\), HR policies and institutional guidelines\. However, retrieval\-augmented generation \(RAG\) approaches that rely on free\-form rewriting can introduce hallucinations and unstable trade\-offs between completeness and conciseness, particularly in safety\- and compliance\-critical settings\.

Objectives:To evaluate extraction as a hallucination\-resistant alternative to rewriting\-based RAG, and to compare strategies that balance precision, recall and safety across document types and model scales\.

Methods:We compare multiple prompting strategies, including line\-number\-based source selection, extraction of relevant guideline sentences with explicit annotation of safety\-related content, and a multi\-stage pipeline that refines draft answers using supporting evidence from the source guideline\. All approaches outperform direct prompting that relies on the model to copy relevant text from the source document\. Experiments are conducted on documents of varying length \(local NHS guidelines in acute care and oncology, as well as UK\-wide NICE guidelines\), and structure using both frontier\-scale and smaller locally deployable models\. Performance is assessed using automatic metrics and via human expert evaluation of relevance and completeness\.

Results:Line\-number selection achieves the strongest results outperforming direct copying and safety\-focused strategies across both large and small models, while maintaining high term recall \(up to 95%\) and close alignment with source text\. Safety\-oriented approaches improve precision but introduce systematic omissions, and multi\-stage filtering further amplifies this trade\-off\. Performance varies with document structure: line\-based extraction excels in protocol\-like content, whereas alternative strategies show advantages in more verbose documents \(up to 97% term recall\)\.

Conclusions:Line\-number selection provides a robust alternative to rewriting\-based RAG for accessing organisational documentation\. Drafting\-based approaches may offer additional benefits in very long\-document settings\. These findings highlight the importance of adapting retrieval strategies to document structure, model scale and the risks of omission\.

## Introduction

Administrative and operational documents, including clinical guidelines, standard operating procedures \(SOPs\), and organisational policies, provide consensus\-based instructions for decision\-making and task execution\[[9](https://arxiv.org/html/2606.12897#bib.bib23)\]\. Across domains, adherence to such documents is a core requirement of good professional practice and is associated with improved safety, compliance, and operational consistency\[[22](https://arxiv.org/html/2606.12897#bib.bib21),[12](https://arxiv.org/html/2606.12897#bib.bib22),[5](https://arxiv.org/html/2606.12897#bib.bib24),[27](https://arxiv.org/html/2606.12897#bib.bib25),[1](https://arxiv.org/html/2606.12897#bib.bib28),[17](https://arxiv.org/html/2606.12897#bib.bib29),[3](https://arxiv.org/html/2606.12897#bib.bib30)\]\.

In practice, high\-level guidance is complemented by local or departmental documents that adapt recommendations to specific workflows, constraints, and organisational structures\. Users consult these resources to answer focused, task\-oriented questions, for example, what actions to take, which approvals are required, or how to escalate a situation\. However, their use is often hindered by complexity and time pressure\[[21](https://arxiv.org/html/2606.12897#bib.bib32),[25](https://arxiv.org/html/2606.12897#bib.bib34),[7](https://arxiv.org/html/2606.12897#bib.bib35),[26](https://arxiv.org/html/2606.12897#bib.bib42)\], challenges that are amplified by increasing workload and growing document volume\[[24](https://arxiv.org/html/2606.12897#bib.bib36),[4](https://arxiv.org/html/2606.12897#bib.bib37),[10](https://arxiv.org/html/2606.12897#bib.bib38)\]\. In high\-stakes or time\-critical settings, these limitations can lead to delays, errors, or non\-adherence\.

Large Language Models \(LLMs\) offer a promising approach to document\-based question answering \(Q&A\)\[[23](https://arxiv.org/html/2606.12897#bib.bib18)\], but their deployment is constrained by hallucinations, i\.e\. generated content not supported by source evidence\[[20](https://arxiv.org/html/2606.12897#bib.bib17)\]\. Retrieval\-Augmented Generation \(RAG\) mitigates this by grounding responses in retrieved documents\[[14](https://arxiv.org/html/2606.12897#bib.bib16)\], improving factual consistency\[[18](https://arxiv.org/html/2606.12897#bib.bib13)\]\. However, it does not fully eliminate errors: models may omit critical steps, introduce irrelevant details, or fail to faithfully reproduce source text, posing risks in safety\- and compliance\-sensitive contexts\[[16](https://arxiv.org/html/2606.12897#bib.bib15),[2](https://arxiv.org/html/2606.12897#bib.bib14),[12](https://arxiv.org/html/2606.12897#bib.bib22),[13](https://arxiv.org/html/2606.12897#bib.bib12)\]\.

A central challenge in document\-based Q&A is balancing interpretation with faithful reproduction\. Some queries require integrating information distributed across multiple sections or adapting conditional instructions to a specific scenario\. Others demand precise, verbatim reproduction to preserve meaning, particularly where exact wording carries operational or regulatory significance\.

Current approaches focus on improving generation quality, rather than controlling when generation should occur\[[12](https://arxiv.org/html/2606.12897#bib.bib22),[13](https://arxiv.org/html/2606.12897#bib.bib12)\]\. Here, we take the opposite approach: we minimise generation\. We constrain models to operate directly on source text through extraction, enabling them to identify and present relevant information without rewriting\. This reframes the task from answer generation to evidence selection, reducing opportunities for hallucination while preserving fidelity to source material\.

Our main contribution is the development and evaluation of an extraction framework for the use case of clinical question answering that minimises hallucinations while balancing the risk of adding information versus omitting clinically critical details\. In contrast to rewriting\-based approaches, we constrain LLMs to select and copy guideline text, targeting high fidelity to source material, an essential requirement in settings where even small inaccuracies in wording \(e\.g\., drug dosing, contraindications, or escalation criteria\) may impact patient safety\. We evaluate this approach across multiple guideline use cases that vary in length and structure, including concise acute care protocols from University College London Hospital \(UCLH\), oncology\-focused institutional guidelines from Somerset NHS Trust, and large\-scale national recommendations \(NICE\)\. These settings capture a spectrum of real\-world decision\-making scenarios\. Evaluation is conducted using human expert\-developed question–answer pairs grounded in these documents, reflecting practical clinical information needs under time pressure\. We further assess performance across a range of LLM configurations, including both open\-weight and commercial models, to characterise robustness across deployment settings\. Rather than attempting to make LLMs better generators, we argue that constraining them offers a more reliable path for deployment in safety\-critical clinical environments\.

## Methods

### Ethical Considerations

This study was approved by the UCL Review Board as TRE\-467283, and approved as a clinical service audit at the National Hospital for Neurology and Neurosurgery, University College London Hospital \(UCLH\) as 130\-202425\-SE\. The guideline data used in this study contained no patient\-level or personally identifiable information\. All materials were derived from institutional clinical guidelines intended for professional use and therefore posed no risk to patient privacy\.

### Data

We evaluate our methods across three datasets spanning distinct document styles and scales: concise acute care guidelines \(UCLH\), moderate\-length institutional guidelines \(Somerset NHS Trust\), and large\-scale national recommendations \(NICE\)\.

The UCLH dataset consists of 96 clinically grounded questions grouped under 24 acute care topics, designed by FJ and NJ to reflect real\-world scenarios encountered by frontline clinicians\. Topics include, for example,acute coronary syndrome, where questions probe escalation decisions \(e\.g\., “If first\-line therapy does not relieve symptoms, when is a second dose indicated?”\), andacute kidney injury, focusing on investigation and management steps \(e\.g\., “Which blood tests are essential on admission?”\)\. Reference answers were constructed via direct selection of relevant guideline lines to ensure grounding in source material\.

The Somerset NHS Trust dataset follows a similar question design process but differs in annotation strategy\. Here, reference answers were generated exclusively through clinician free writing \(NS and FJ\)\. These guidelines are predominantly oncology\-oriented and capture acute complications of cancer and its treatment, such asneutropenic sepsis,metabolic disturbances, andtreatment\-related toxicities\. Despite being moderate in length, they are strongly acute in nature, often focusing on time\-critical recognition, escalation, and stabilisation decisions in vulnerable patient populations\.

In contrast, the NICE dataset was generated synthetically following the protocol of Ding et al\.\[[6](https://arxiv.org/html/2606.12897#bib.bib3)\], usinggpt\-4o\-minito produce clinical scenario–question–guideline paragraph triplets from full\-length recommendation documents\. To reflect realistic model inputs, we merge the scenario description and associated question into a single input prompt\. All generated outputs were subsequently manually reviewed by human experts to ensure clinical validity\. This dataset captures scenario\-related inputs, for example withinchronic kidney diseaseorheart failuremanagement\. A representative example is:

“A 68\-year\-old patient with a history of hypertension and type 2 diabetes presents with progressive fatigue and ankle swelling over several weeks\. Blood tests show reduced eGFR and elevated creatinine\. According to current guidelines, what investigations and initial management steps should be undertaken, and when should specialist referral be considered?”

This contrasts with the shorter, focused queries in UCLH and Somerset, such as:“Which blood tests are essential in acute kidney injury?”\(see TablesLABEL:tab:task\_questions\_longandLABEL:tab:oncology\_questions\)\.

All underlying guideline documents were extracted from PDF files using thepdftotextutility to preserve original wording, followed by manual cleaning to remove tables of content, correct formatting artefacts and ensure fidelity to source content\.

Across datasets, question formulation varies substantially, from concise queries in UCLH \(mean 9\.6 words\) to longer, scenario\-based inputs in NICE \(mean 45\.9 words\), with Somerset NHS Trust occupying an intermediate position\. Importantly, the guideline corpora differ by an order of magnitude in length: short UCLH documents \(mean 746 words\), moderate\-length Somerset NHS Trust guidelines \(mean 3,153 words\), and long\-form NICE recommendations \(mean 5,224 words; maximum 8,007 words\) \(Table[1](https://arxiv.org/html/2606.12897#Sx2.T1)\)\.

Table 1:Summary statistics of the evaluation datasets spanning three document\-length regimes: concise acute care guidelines \(UCLH\), moderate\-length institutional guidelines \(Somerset NHS Trust\), and large\-scale national recommendations \(NICE\)\. Values are reported as mean \(min–max\)\.
### Prompting Strategies for Guideline\-Based Clinical Question Answering

This section outlines the methods evaluated in this study, starting with a baseline retrieval approach based on semantic similarity between the user question and guideline text\. We then present a set of prompting pipeline methods that combine an initial topic identification step with different prompting regimes for extracting relevant evidence from clinical guidelines \(see Fig\.[1](https://arxiv.org/html/2606.12897#Sx2.F1), see summary of the approaches in Table[2](https://arxiv.org/html/2606.12897#Sx2.T2)\)\.

![Refer to caption](https://arxiv.org/html/2606.12897v1/Figures/pipeline_prompting_framework.png)Figure 1:Overview of the proposed guideline question\-answering pipeline\. Given a user question, the system first performsTopic Selectionto identify the most relevant clinical guideline from the guideline index\. Once the relevant guideline is selected, the system applies one of severalprompting strategiesto extract the evidence required to answer the question\. The evaluated strategies include: \(i\)COPY, where the model extracts verbatim sentences; \(ii\)LINES, where the model selects relevant guideline line numbers; \(iii\)JuniorDoc, a multi\-agent workflow where a junior\-doctor\-style draft answer is generated, verified against the guideline, and filtered by a safety editor; and \(iv\)Safety, which extracts the guideline sentences relevant to the question and annotates safety\-critical content to support safe evidence\-based responses\.##### Baseline Retrieval \(BASE\)

As a baseline retrieval strategy, we construct a repository of information chunks derived from guideline text, each representing a self\-contained unit of clinically relevant content that may be used to answer a question\. From each guideline, we extract guideline sections and, for each section, the section title, section text, and a representative actionable question\.

These sections are extracted automatically from guideline paragraphs using an LLM prompt\. This section extraction prompt \(Listing[1](https://arxiv.org/html/2606.12897#LST1)\) decomposes each paragraph into smaller, clinically meaningful overlapping subparts\. For each section, it then generates a short descriptive title and an actionable clinical question\. By associating each text segment with a representative question, we aim to reduce sensitivity to phrasing variation between guideline text and associated queries\.

At indexing time, each subpart is embedded using one of two representations \(Figure[2](https://arxiv.org/html/2606.12897#Sx2.F2)\): \(a\)text, based on guideline topic, section title, and section text concatenated into single string; or \(b\)text \+ question, which additionally incorporates the representative question \(guideline topic, section title, section text and representative question concatenated into single string\)\. At query time, the user question is embedded using the same model and compared to all embeddings \(variant \(a\) or \(b\)\) via cosine similarity\. Retrieval is then performed by selecting the subparts whose representations are most semantically aligned with the query\.

![Refer to caption](https://arxiv.org/html/2606.12897v1/Figures/baseline_index_pipeline.png)Figure 2:Construction of the representative question–section index used by the baseline retrieval strategies \(BASE\)\. Each guideline document is first segmented into paragraphs, from which clinically coherent overlapping sections are extracted\. For every section, a representative question and section title are generated using an LLM\. Each subpart is represented by four elements: the guideline topic, section title, representative question, and verbatim section text\. Two indexing variants are considered: \(a\)text, where embeddings are computed from guideline topic, section title and text concatenated together into a single string; and \(b\)text \+ question, where embeddings are computed as in \(a\) plus the representative question\. Representative questions act as a semantic interface between guideline text and user queries\. At query time, the user question is embedded using the same embedding model and compared against section embeddings via cosine similarity\.Table 2:Summary of prompting strategies\. All methods begin with Topic Selection, which identifies the most relevant guideline document before extraction\.
##### Step 1: Topic Selection \(Topic\)

Topic selection is the first step across all strategies\. Given a user query, the model selects exactly one guideline topic from a predefined index of topics\. This design follows the workflow observed in clinical practice\. Questions typically relate to a single diagnosis, meaning clinicians first identify the relevant guideline before consulting detailed sections \(see Listing[2](https://arxiv.org/html/2606.12897#LST2)\)\.

Since our study focuses on acute medical problems encountered on an undifferentiated hospital ward, clinical questions can generally be mapped to a single primary issue, even for patients with substantial multimorbidity\. Furthermore, many acute conditions, such as hyperkalaemia or hypoglycaemia, are managed according to standard protocols that are largely independent of a patient’s underlying comorbidities\. Where management does require adaptation for specific patient groups, such as those with chronic kidney disease, these considerations are typically captured within dedicated sections of the same guideline\. This structure supports a topic\-first retrieval strategy, in which a single guideline is identified before relevant subsections are selected\.

##### Step 2: Evidence Retrieval Strategies

##### Verbatim Sentence Strategy \(COPY\)

We first introduceCOPYas a prompting baseline that probes the model’s ability to directly copy relevant information from the guideline\. The model is instructed to return only verbatim sentences that answer the user question, with explicit constraints forbidding paraphrasing and requiring exact reproduction of the source text\.

This setup probes models’ capacity for faithful extraction, providing a reference point for more constrained strategies \(see Listing[4](https://arxiv.org/html/2606.12897#LST4)\)\.

##### Line Selection Strategy \(LINES\)

Building on this baseline, theLINESstrategy introduces a more sophisticated mechanism\. The guideline text is augmented with line numbers and provided to the model together with the user question\. The model is then tasked with identifying the minimal set of line numbers required to answer the question\.

The final answer is reconstructed deterministically by retrieving the corresponding guideline lines in post\-processing\. This approach enforces tighter control over the extracted content and guarantees the absence of hallucinated text, as the output consists exclusively of selected guideline line numbers \(see Listing[3](https://arxiv.org/html/2606.12897#LST3)\)\.

##### Junior Doctor Workflow \(Junior\)

We also design a multi\-agent prompting strategy that mimics a realistic clinical reasoning workflow\.

Agent 1 acts as a junior doctor and generates an initial answer based only on training memory, without consulting external documents\. This simulates the real\-world setting in which clinicians often rely on prior knowledge under time pressure\. The agent is instructed to provide concise answers containing actionable details such as thresholds, medication doses, or monitoring steps \(see Listing[5](https://arxiv.org/html/2606.12897#LST5)\)\. Agent 2 then compares the draft answer against the official guideline text and selects the guideline sentences that best support, clarify, or correct it, preserving their original order \(see Listing[6](https://arxiv.org/html/2606.12897#LST6), see Figure[3](https://arxiv.org/html/2606.12897#Sx2.F3)\)\.

![Refer to caption](https://arxiv.org/html/2606.12897v1/Figures/juniordoc_example.png)Figure 3:Illustration of theJuniorDocprompting workflow\. The user question is first processed byAgent 1, which acts as a junior clinician reasoning from memory and produces an initial draft answer\. This draft may be incomplete or imprecise because it did not yet consult the guideline text\.Agent 2then grounds the answer by extracting verbatim sentences from the relevant clinical guideline section that support, refine, or correct the initial draft\. This multi\-stage design combines clinical reasoning and evidence grounding\.
##### Minimal Safety Extraction \(Safety\)

We also evaluate a prompting strategy that instructs the model to extract the minimal safe set of sentences required to answer the user’s question\. The model is prompted to select the smallest subset of full sentences that \(i\) directly answer the question and \(ii\) include any safety\-critical information\. Each selected sentence is annotated with a single reason label indicating its role in the response:answers\_question,adds\_required\_detail, orsafety\_critical\. This prompt also enforces strict constraints: sentences must be copied verbatim from the guideline, paraphrasing is not allowed, and the original order must be preserved\.

The Safety strategy is motivated by the observation that guideline answers often contain both essential recommendations and surrounding explanatory text\. By explicitly defining roles, the model is guided to retain only the most relevant content\.

### LLM Types

To evaluate the robustness of our methods across different model architectures and deployment settings, we conducted experiments using several open\-weight LLMs available through the Ollama framework111[https://ollama\.com](https://ollama.com/)\. Specifically, we evaluatedgpt\-oss\-120b\[[19](https://arxiv.org/html/2606.12897#bib.bib10)\]andqwen3:32b\[[28](https://arxiv.org/html/2606.12897#bib.bib9)\]\. These models provide openly available weights, enabling reproducibility and supporting deployment in secure environments where external API access may be restricted \(e\.g\., clinical or trusted research environments\)\. Using open\-weight models also allows evaluation under controlled and identical inference conditions\. All models were run with default hyperparameters except for the temperature parameter, which was fixed at 0 to minimise non\-deterministic variation\.

For the baseline retrieval strategy \(BASE\), we additionally experimented with three embedding models available through Ollama:nomic\-embed\-text,embeddinggemma, andqwen3\-embedding\. These models differ primarily in their training objectives and underlying model families\.nomic\-embed\-textis designed specifically for semantic search and retrieval tasks,embeddinggemmaprovides lightweight embeddings derived from the Gemma model family\[[8](https://arxiv.org/html/2606.12897#bib.bib6)\], andqwen3\-embeddingis based on the Qwen architecture and trained to produce dense representations aligned with instruction\-tuned models\.

For the open\-access NICE guideline data, we additionally evaluate large\-scale API\-based models to assess how our extraction strategies generalise to frontier models beyond locally deployable setups\. Specifically, via the Replicate platform, we include models spanning a wide scale range:deepseek\-ai/deepseek\-v3\(very large, 671B parameters\),qwen/qwen3\-235b\-a22b\-instruct\-2507\(medium, 235B parameters\), andgpt\-5o\-mini\(compact, parameter count undisclosed, typically <30B\)\. We deliberately excludegpt\-4o\-minifrom this evaluation, as it was involved in the creation of the synthetic NICE dataset\.

### Evaluation

We evaluate the proposed methods using both automatic metrics and human assessment\.

##### Automatic Evaluation

We first report average answer length \(AnsLen\) to understand model verbosity, as longer outputs tend to increase recall\-oriented metrics while potentially reducing precision\.

We then evaluate overlap with reference answers using ROUGE\-L\[[15](https://arxiv.org/html/2606.12897#bib.bib7)\], which can be interpreted simply as a form ofsentence\-level recall: it measures how much of the reference content is recoveredin the correct order, rewarding correct reconstruction of long chunks of text rather than fragmented matches\.

To capture more fragmented lexical overlap, we report word\-level precision, recall, and F1 \(P, R, F1\)\. Both reference and generated answers are normalised by removing non\-alphanumeric characters, lowercasing, and splitting into tokens on space\. Precision is defined as the proportion of generated tokens supported by the reference, recall as the proportion of reference tokens recovered, and F1 as their harmonic mean\. This formulation is robust to word order and minor formatting differences\.

Finally, we assess entity\-level fidelity using medical named entity recognition \(NER, medical term\) precision, recall, and F1\.

We use MedCAT\[[11](https://arxiv.org/html/2606.12897#bib.bib1)\]for biomedical named entity recognition and concept linking\. Specifically, we employ the latest available MedCAT model packs based on SNOMED CT and UMLS, accessed through UMLS\-authenticated releases\.222[https://medcat\.readthedocs\.io/en/latest/main\.html](https://medcat.readthedocs.io/en/latest/main.html)MedCAT identifies clinical concepts in guideline texts and maps them to standard biomedical terminologies\. Evaluation is performed over sets of unique extracted term spans, with precision and recall defined as the overlap between predicted and reference entity sets, and F1 as their harmonic mean\. This provides a targeted measure of whether clinically salient concepts \(e\.g\., drugs, conditions, symptoms\) are correctly preserved, independent of surrounding text\.

Together, these metrics provide complementary views of extraction quality: AnsLen captures verbosity, ROUGE\-L reflects sentence\-level overlap, word\-level scores capture fragmented lexical overlap, and medical term metrics focus specifically on clinically relevant entities\.

Table 3:Automatic evaluation metrics highlighting complementary aspects of verbosity, coverage and clinical fidelity\.
##### Human Evaluation

In addition to automatic metrics, we conducted a human evaluation to assess the clinical quality of the generated answers\. While automatic metrics quantify textual overlap, they are limited in their ability to capture clinical correctness, contextual appropriateness, and the safety implications of missing or added information\. In guideline\-based question answering, small deviations in wording, omission of key recommendations, or inclusion of irrelevant details may not substantially affect metric scores but can have significant clinical consequences\.

Three authors \(FJ, EG, and NS; only one of those authors participated in the UCLH guideline data creation\) independently reviewed the outputs while having access to the original guideline PDF documents used as source material\. Importantly, the evaluators did not have access to the reference answers used in the automatic evaluation\. They assessed each system output directly against the guideline documents\.

The evaluators scored answers along two dimensions\. The first criterion wascorrectness, indicating whether the information provided was relevant to the clinical question \(categories:very relevant,relevant,irrelevant\)\. The second criterion wascompleteness, indicating whether the answer adequately covered the relevant guideline content \(categories:minor omission,major omission,just right,minor addition,major addition\)\. Major omissions are particularly important in clinical contexts because missing key recommendations may lead to incomplete or unsafe guidance\.

## Results

Table 4:Comparison of index characteristics across datasets\. UCLH \(acute care\) shows concise, action\-oriented structure; Somerset \(oncology\) is more narrative; NICE \(national guidance\) is highly heterogeneous\.StatisticUCLHSomersetNICENumber of guidelines24610Number of sections9874694,726Sections per guideline, mean41\.178\.2472\.6Sections per guideline, range6–7448–12512–849Section length, words mean31\.465\.818\.6Question length, words mean8\.78\.28\.3##### Baseline Index Construction

Applying the baseline indexing procedure \(see Methods\) across the three data sources yields corpora reflecting differences in each guideline style\.

For the UCLH dataset, the indexing process produced 987 sections\. Sections are relatively short \(mean 31 words\), guidelines are decomposed into an average of 41 sections \(see Table[4](https://arxiv.org/html/2606.12897#Sx3.T4)\)\. In the Somerset NHS Trust dataset, the index contains 469 sections across 6 guidelines \(mean 78 sections per guideline\)\. Sections are substantially longer than in UCLH \(mean 66 words\), reflecting a more narrative style with multiple clinical considerations rather than concise decision rules\. The NICE dataset is substantially larger and more heterogeneous, comprising 4,726 sections across 10 guidelines \(mean 473 sections per guideline\)\. Sections are shorter on average \(mean 19 words\)\. Across all datasets, representative questions remain short and consistent \(≈\\approx8 words\)\. Examples are shown in Table[A3](https://arxiv.org/html/2606.12897#Sx12.T3)\. These questions capture key clinical core reasoning patterns, including diagnostic criteria, escalation decisions, and safety\-critical alerts\.

Table 5:Automatic evaluation of baseline \(BASE\) retrieval across embedding models and representations\. We compare representations based on section text, section topic and title \(text only\) and representations based on section text, section topic, title and representative LLM\-generated question \(text \+ question\)\. Guideline sections are retrieved using cosine similarity between embedded user query and representations, and returned in full as answers\. Metrics include ROUGE\-L \(R\-L, sentence\-level recall\), word\-level precision \(P\), recall \(R\), and F1, as well as medical term precision, recall, and F1\. Answer length \(AnsLen\) is reported as mean word count \(lower is better\)\. BASE achieves consistently high recall across embeddings, but low precision and long outputs, indication non\-essential content\. Best results per representation are shown inbold, second best are underlined\.Table 6:Embedding similarity statistics for baseline retrieval\. Values represent cosine similarity between the query and the top\-ranked guideline section\. We compare representations based on section text, section topic and title \(text only\) and representations based on section text, section topic, title and representative LLM\-generated question \(text \+ question\)\. Results show overall high similarity across models, withnomicachieving the highest mean similarity and lowest variance, andgemmashowing lower similarity\. Best values per representation are shown inbold\.
##### Baseline retrieval limitations\.

Our BASE retrieval strategy provides an upper bound for content coverage, consistently achieving high recall across all configurations \(word\-level recall=0\.65=0\.65–0\.860\.86; Table[5](https://arxiv.org/html/2606.12897#Sx3.T5)\)\. This indicates that relevant guideline content is retrieved within the selected sections\. However, precision remains substantially lower \(word precision=0\.12=0\.12–0\.240\.24\), reflecting that retrieved sections frequently include non\-essential content\.

BASE produces long outputs \(mean 174–257 words\), as entire guideline sections are returned rather than selectively extracted evidence\. This reduces usability of those answers in time\-critical clinical settings\.

Across embedding strategies,qwen3achieves the strongest performance overall\. In both representations, it yields the highest ROUGE\-L and F1 scores, while also producing the shortest outputs among BASE variants \(174 words for text\-only\)\.

Augmenting section representations with LLM\-generated questions \(text \+ question\) produces mixed effects across embeddings\. In particular, it is beneficial forqwen3andnomic, but decreases performance forgemma\. Also, text \+ question representations tend to slightly increase output length\.

All models achieve relatively high cosine similarity \(mean0\.650\.65–0\.760\.76, Table[6](https://arxiv.org/html/2606.12897#Sx3.T6)\), confirming that retrieval identifies semantically relevant sections\.gemmaexhibits lower mean similarity and higher variance, consistent with its weaker downstream retrieval performance\.

Table 7:Performance comparison on the UCLH dataset across prompting\-based extraction strategies and model families\. Metrics include ROUGE\-L \(R\-L, sentence\-level recall\), word\-level precision \(P\), recall \(R\), and F1, as well as medical term precision, recall, and F1\. Answer length \(AnsLen\) is reported as mean word count \(lower is better\)\. Statistical significance is assessed via paired bootstrap resampling \(10,000 draws with replacement\), with 95% confidence intervals and two\-sided p\-values computed from the bootstrap distribution; significance \(\*\) indicatesp<0\.05p<0\.05relative to theCOPYbaseline\.LINESconsistently achieves the highest recall among prompting methods, approaching the coverage of baseline retrieval while producing substantially shorter outputs\. Bold highlights the top\-performing methods per metric, second\-best performance is underlined\.Table 8:Automatic evaluation of prompting strategies for GPT\-OSS\-120B with full Somerset NHS guideline context\. Metrics include ROUGE\-L \(R\-L, sentence\-level recall\), word\-level precision \(P\), recall \(R\), and F1, as well as medical term precision, recall, and F1\. Answer length \(Len\) is reported as mean word count \(lower is better\)\. Statistical significance is assessed via paired bootstrap resampling \(10,000 draws with replacement\), with 95% confidence intervals and two\-sided p\-values computed from the bootstrap distribution; significance \(\*\) indicatesp<0\.05p<0\.05relative to theCOPYbaseline\. Results highlight a clear precision–recall trade\-off\.Juniorachieves the highest recall\. Bold indicates the best\-performing method per metric, and second\-best values are underlined\.Table 9:Automatic evaluation of prompting strategies across model families on full NICE guidelines\. Metrics include ROUGE\-L \(R\-L, sentence\-level recall\), word\-level precision \(P\), recall \(R\), and F1, as well as medical term precision, recall, and F1\. Answer length \(Len\) is reported as mean word count \(lower is better\)\. Statistical significance is assessed via paired bootstrap resampling \(10,000 draws with replacement\), with 95% confidence intervals and two\-sided p\-values computed from the bootstrap distribution; significance \(\*\) indicatesp<0\.05p<0\.05relative to theCOPYbaseline\. In this setting with very long guidelines \(over 5,000 words\),Juniorconsistently achieves the highest recall across models \(e\.g\., word recall up to∼\\sim0\.94\), with statistically significant improvements overCOPY, while maintaining shorter outputs thanLINES\. Bold indicates the best\-performing method per metric, and second\-best values are underlined\.
##### Prompting\-based extraction strategies\.

Across all datasets and model families, prompting\-based extraction substantially improves over baseline retrieval, not by retrieving more information, but by selecting it more precisely\. Among these approaches for UCLH,LINESconsistently emerges as the most reliable strategy for preserving clinically relevant content \(Table[7](https://arxiv.org/html/2606.12897#Sx3.T7)\)\.

UnlikeCOPY, which serves as a baseline for verbatim extraction,LINESexplicitly enforces line\-level selection of evidence\. This constraint proves critical\. Across settings,LINESachieves the highest recall across both word\-level and entity\-level metrics \(e\.g\., word recall0\.940\.94and NER recall0\.950\.95for GPT\-OSS;0\.900\.90and0\.910\.91for Qwen\), with statistically significant improvements overCOPY\. In safety\-critical settings, where missing information may have direct consequences, this recall advantage is decisive\.LINEStherefore provides the most effective balance of completeness and usability\.

COPYprovides a strong precision\-oriented baseline\.Safetybehaves very similarly toCOPY, achieving precision close toCOPYacross settings, while producing outputs of similar length\.Juniorimproves recall relative to other methods, but consistently fall short ofLINESwhile producing longer and less precise responses\.

![Refer to caption](https://arxiv.org/html/2606.12897v1/Figures/pie_majority_whole_csv_correctness_completeness.png)Figure 4:Clinician evaluation of LINES outputs\. Distribution of relevance \(left\) and completeness \(right\) ratings across 96 clinical questions\. LINES achieves high clinical relevance \(95\.8% at least relevant\) and the highest proportion of “just right” answers \(62\.5%\), indicating strong alignment with reference guidance\. No major omissions were observed, suggesting that LINES reliably preserves critical clinical content\.![Refer to caption](https://arxiv.org/html/2606.12897v1/Figures/pie_majority_whole_csv_correctness_completeness_safety.png)Figure 5:Clinician evaluation of Safety outputs\. Distribution of relevance \(left\) and completeness \(right\) ratings across 96 clinical questions\. Safety produces highly relevant outputs \(93\.7% at least relevant\), with a higher proportion of “very relevant” ratings compared to LINES, reflecting concise and focused responses\. The strategy presents increased omissions \(35\.5%, including 16\.7% major omissions\)\.
##### Extraction behaviour analysis\.

Table[A4](https://arxiv.org/html/2606.12897#Sx12.T4)provides qualitative insight into how different strategies work\. For example, for Pericarditis Q10, the reference answer specifies three key elements: chest radiograph, transthoracic echocardiogram, and cardiology input\. Both LINES and COPY recover all elements, while Safety keeps only the mimimum subset\.

For Q11 Pericarditis, LINES most closely matches the reference answer with all key elements\. COPY also achieves high coverage but introduces minor omission of the cardiology follow\-up\. In contrast, the Safety strategy produces a more concise output, prioritising core treatment steps but omits important supporting details, notably PPI co\-prescription, GI side\-effect management, and cardiology follow\-up\. These trends are consistent across both GPT\-OSS and Qwen models\.

Qualitative analysis of JuniorDoc \(Table[A5](https://arxiv.org/html/2606.12897#Sx12.T5)\) shows that Agent\-2 effectively consolidates key guideline elements\. However, additions of non\-essential contextual information and omission of relevant details are frequent\. For example, in AF, Agent\-2 captures the core decision rule for cardioversion \(onset<<48 h with no precipitant\), but introduces additional context \(broader rhythm control pathways\)\.

##### Performance of extraction strategies for longer documents and frontier models\.

Overall, the performance of our extraction strategies depends strongly on document length and retrieval setting\. When applied to full, long\-form guidelines \(Somerset and NICE\), the JuniorDoc multi\-agent approach becomes increasingly competitive, and in most of the cases achieves the highest recall across models \(Tables[8](https://arxiv.org/html/2606.12897#Sx3.T8)and[9](https://arxiv.org/html/2606.12897#Sx3.T9)\)\. For example, on full NICE guidelines \(often exceeding 5,000 words\), JuniorDoc consistently achieves the strongest word\-level and medical term recall across DeepSeek, GPT\-5\-mini, and Qwen3, with statistically significant improvements overCOPY\.

This behaviour may be attributed to the prompt pipeline: agent 1 generates a response draft aligned with clinicians’ ideas about the problem, while agent 2 retrieves supporting guideline evidence\. In long documents, this enables the model to pinpoint evidence that would be difficult to capture in this long context\. As a result, JuniorDoc is well\-suited to large guidelines such as Somerset \(∼\\sim3,000 words\) and NICE \(∼\\sim5,000 words\)\.

The pattern reverses when retrieval is constrained\. When guideline sections are pre\-selected \(e\.g\., via cumulative similarity mass thresholding\),LINESconsistently achieves the highest recall across models while maintaining moderate answer length \(Tables[A6](https://arxiv.org/html/2606.12897#Sx12.T6)and[A7](https://arxiv.org/html/2606.12897#Sx12.T7)\)\. In this setting, the search space is reduced to a focused subset of relevant content \(e\.g\., for NICE from∼\\sim5,000 words per full guideline to∼\\sim800 words per selected sections\)\.

##### Human Evaluation

Following strong automatic performance, we conducted a detailed human evaluation of model outputs generated by the GPT\-OSS\-120B model across 96 UCLH clinical questions, comparing both the LINES and Safety strategies \(Figures[4](https://arxiv.org/html/2606.12897#Sx3.F4)and[5](https://arxiv.org/html/2606.12897#Sx3.F5)\)\. The Safety strategy was additionally included as a controlled contrast in output length\. This contrast provides a useful lens for assessing whether metric gains reflect true clinical quality\. Clinical review was performed by three clinicians \(FJ, EG, NS\), assessing both relevance and completeness\. Relevance was rated asirrelevant,relevant, orvery relevant, reflecting appropriateness to the clinical scenario\. Completeness captured whether clinically important information was omitted or whether non\-essential content was included\.

Relevance\.Both strategies produced highly clinically appropriate outputs\. For LINES, 69\.8% of responses were rated asvery relevantand 26\.0% asrelevant, with only 4\.2% classified asirrelevant\(4 answers\)\. Safety achieved even higher top\-tier relevance, with 85\.4% of outputs rated asvery relevantand 8\.3% asrelevant, though a slightly higher proportion \(6\.2%\) were judgedirrelevant\.

Completeness\.Differences appear more clearly in completeness\. LINES produced the highest proportion ofjust rightanswers \(62\.5%\), with relatively few omissions \(8\.3% minor, 0% major\), but more frequent additions \(22\.9% minor, 6\.2% major\), reflecting its tendency to include additional contextual detail\. In contrast, Safety produced fewer additions \(12\.5% minor, 1\.0% major\) and a lower proportion of excessive content overall, but at the cost of increased omissions \(18\.8% minor, 16\.7% major\)\. Only 51\.0% of Safety outputs were rated asjust right\.

Overall, LINES prioritises completeness and coverage, rarely omitting critical information but sometimes including non\-essential content\. Safety enforces brevity and reduces over\-generation, but more frequently omits clinically relevant details, including in some cases information that may be important for safe decision\-making\.

Illustrative examples\.The pancreatitis example \(see Table[A8](https://arxiv.org/html/2606.12897#Sx12.T8)\) demonstrates a minor addition for LINES\. The response appropriately prioritises urgent referral to ICU outreach, including clear clinical criteria to support severity assessment and escalation decisions\. It also provides relevant contact pathways, including the general medical registrar, ICU outreach services, hepatobiliary specialists, and dietetics, all of which are appropriate for ongoing care\. The inclusion of additional sections on ongoing management and imaging is not directly required to answer the referral question\. However, this information remains clinically relevant and does not detract from decision\-making\.

The ureteric colic example illustrates a minor omission\. The response provides a clear and clinically useful set of discharge criteria that are easy to interpret and apply\. One element of the guideline, relating to stone size and anatomical location, is not explicitly included and would require reference to an associated table\. This omission does not significantly affect clinical decision\-making, as it can be readily clarified if needed and does not compromise patient safety\.

Inter\-annotator agreement\.Inter\-annotator agreement was assessed on a subset of 12 shared questions \(Pericarditis, Acute Kidney Injury, and Atrial Fibrillation\)\. For relevance, unanimous agreement was observed in 3 cases, with the remaining 9 showing agreement between two annotators\. Disagreements were limited to adjacent categories \(e\.g\.relevantvsvery relevant\)\. For completeness, full agreement was observed in 6 cases, with the remaining 6 showing partial agreement\. Disagreements again occurred primarily between neighbouring categories, most commonlyjust rightandaddition–minor\. Majority voting was used to assign final labels\.

These findings confirm that theLINESstrategy provides a strong mechanism for clinically evidence extraction\. The consistently high relevance \(95\.8%\) and absence of major omissions indicate thatLINESeffectively preserves critical safety\-relevant information\. Minor omissions tend to affect peripheral details, confirming thatLINESavoids unsafe under\-specification\.

## Discussion

##### Statement of key findings

Our results demonstrate that clinically relevant information can be reliably delivered by an LLM\-based system with relevant prompting\. A central finding is that the effectiveness of extraction strategies depends critically on document length and retrieval setting\. For shorter protocol\-like guidelines,LINESconsistently achieves the highest term recall across models while maintaining moderate answer length \(\+10 points increase\)\. In this setting, the reduced search space enables precise line\-level selection\.

When operating over long guidelines \(e\.g\., Somerset and NICE\), theJuniorDocapproach becomes more effective and often achieves the highest term recall \(\+7 points on average\)\. This reflects its ability to draft required evidence first and then more efficiently find it in long documents\.

These results highlight a fundamental trade\-off between completeness of extractive approaches and conciseness of abstractive approaches\. WhileLINESprovides the highest information recall for shorter documents, draft\-first approachesJuniorDocare better suited to large\-context settings\. Strategies that re\-interpret input text \(e\.g\.,CopyandSafety\) improve brevity but may omit clinically relevant information, hence raising safety concerns\.

##### Strengths and limitations

This study introduces an approach to clinical question answering in which LLMs are constrained to operate as evidence extractors rather than free\-text generators\. By enforcing reproduction of guideline content, the approach directly addresses one of the key risks of LLM deployment in healthcare, the risk of hallucination, while enabling transparent and auditable outputs\.

A further strength is the systematic comparison of prompting strategies across different document\-length regimes\. This reveals that performance is not solely a function of model capacity, but of the interaction between prompting strategy and input structure\. In particular, the results demonstrate that increasing architectural complexity \(e\.g\. multi\-agent pipelines\) may not always be beneficial\.

However, several limitations should be mentioned\. First, human evaluation was conducted on a relatively small set of clinical questions, and although designed to reflect realistic scenarios, larger\-scale validation is required\. Second, the study focuses on text\-based guidelines and does not include common tabular or visual elements \(e\.g\. flowcharts\)\. Extending the approach to multimodal guideline formats remains an important direction\.

Third, evaluation was performed in a simulated setting rather than in real\-time clinical workflows\. While this enables controlled comparison, it does not fully capture real\-world usage constraints\. Finally, although we evaluated several models, results may further vary with different architectures and context lengths\. Nevertheless, the consistency of observed patterns across models suggests that the findings are primarily driven by prompting strategy rather than model\-specific effects\.

##### Implications and next steps

These findings have important implications for the deployment of LLMs in clinical settings\. First, they suggest that retrieval and extraction should be jointly designed\. Systems operating over large contexts may benefit from draft\-based approaches such asJunior, whereas systems operating over shorter contexts can rely on extraction methods such asLINES\.

Second, the results emphasise the need to explicitly manage the precision–recall trade\-off\. In safety\-critical applications, prioritising recall may be necessary to avoid omission of clinically relevant information, even at the cost of longer outputs\. Future systems may benefit from adaptive strategies that dynamically adjust verbosity based on task complexity\.

Future work will focus on real\-world deployment, including integration with electronic health record systems and evaluation in live clinical environments\. In addition, interactive capabilities, such as clarifying missing information, may enhance both completeness and relevance\.

## Conclusion

This study demonstrates that LLM\-based systems can provide fast and reliable to guideline information when constrained to evidence extraction\. Crucially, the optimal extraction strategy depends on the retrieval setting: line number extraction is most effective protocol\-like documents, whereas draft\-first approaches are advantageous for long documents\.

Beyond eliminating hallucination, our findings emphasise that preserving completeness is essential for patient safety\. By combining extraction, drafting and retrieval techniques, LLMs can move towards becoming trustworthy clinical assistants\.

## Code Availability

## Data Availability

The clinical guideline data used in this study are proprietary and cannot be publicly shared due to confidentiality requirements\. NICE Q&A dataset is publicly available online:[https://github\.com/julia\-ive/guidelines\_qa](https://github.com/julia-ive/guidelines_qa)

## Competing interests

We have no competing interests to declare\.

## Funding

Not applicable

## Author Contributions Statement

JI: Conceptualisation, Methodology, Software, Validation, Formal analysis, Investigation, Writing – Final draft preparation, Writing \- Reviewing and Editing\. FJ: Conceptualisation, Methodology, Validation, Formal Analysis, Investigation, Writing – Early draft preparation, Writing \- Reviewing and Editing\. EG: Conceptualisation, Methodology, Formal Analysis, Investigation, Writing – Early draft preparation, Writing \- Reviewing and Editing\. NJ: Conceptualisation, Writing \- Reviewing and Editing\. CSH, PB, EC: Writing \- Reviewing and Editing\. RD: Conceptualisation, Methodology, Writing \- Reviewing and Editing\. All authors approved the manuscript\.

## Appendices

Table A1:Clinical scenario questions for human evaluation\.Questions grouped by acute medical topics targeting decision\-critical elements including diagnosis, investigation, treatment, escalation, and discharge safety\.TopicQuestionAcute coronary syndromesShould I ever start ACS treatment before the troponin result is available?Acute coronary syndromesIf I have already sent a troponin sample, when do I send a second one?Acute coronary syndromesDo I give supplementary oxygen to all ACS patients?Acute coronary syndromesMy patient was not on any anticoagulation preadmission\. How long do they need to stay on fondaparinux?Acute diarrheaWhat key discharge advice should I give to outpatients managing acute diarrhoea at home?Acute diarrheaIn an inpatient who develops new diarrhoea \(Bristol 5\+\), what tests and actions are required?Acute diarrheaIs it safe to use anti\-motility agents \(e\.g\. loperamide\) or opiates in acute diarrhoea?Acute diarrheaHow do I decide between oral and IV rehydration?Acute kidney injuryWhich nephrotoxic medications must I stop or adjust immediately?Acute kidney injuryWhat is a ‘full renal screen’?Acute kidney injuryWhen should I involve ICU or renal specialist for possible RRT?Acute kidney injuryHow do I manage pulmonary oedema caused by AKI?AFHow do I decide if cardioversion is safe based on time of onset?AFWhat monitoring and nursing observations are required once therapy is started?AFWhich baseline investigations must I request on initial assessment?AFHow do I risk stratify for bleeding with anticoagulation?Cardiogenic pulmonary oedemaWhat oxygen saturation targets should I aim for, and when is oxygen indicated?Cardiogenic pulmonary oedemaWhich baseline medications should I review or stop immediately?Cardiogenic pulmonary oedemaWhich imaging modalities should I arrange, and how urgently?Cardiogenic pulmonary oedemaWhat is the recommended initial IV diuretic regimen?COPDWhat immediate investigations and samples should I request on arrival?COPDWhich patients warrant ID/Micro review?COPDWhat criteria mandate early ICU referral?COPDWhen should I consider aminophylline?Decompensated liver cirrhosisWhat bowel washout regimen should I start for hepatic encephalopathy?Decompensated liver cirrhosisWhat VTE prophylaxis is appropriate and what are contraindications?Decompensated liver cirrhosisWhat acute hepatic screen should I order?Decompensated liver cirrhosisWho would I call for urgent ascitic drain insertion?DKAOnce stable, what maintenance fluid regimen is recommended?DKAHow often should I monitor CBG/VBG and electrolytes?DKAWhen should I discuss with ICU for escalation?DKAIs there an order set for insulin prescribing?GCAShould I refer to rheumatology or ophthalmology?GCAShould I treat first or discuss first?GCAWhat symptoms suggest complicated GCA?GCABeyond steroids, what treatment is required?Hot swollen jointWhat are the main causes?Hot swollen jointWhat investigations are required?Hot swollen jointHow is septic arthritis treated?Hot swollen jointWhat is first\-line treatment for gout?HypercalcaemiaWhat are the main symptoms?HypercalcaemiaWhat investigations beyond bloods are required?HypercalcaemiaAt what level is it considered severe?HypercalcaemiaWhich drugs commonly cause it?HyperkalaemiaWhat are the severity cut\-offs?HyperkalaemiaWhat is pseudohyperkalaemia?HyperkalaemiaWhat are ECG signs?HyperkalaemiaWhen is cardiac monitoring required?HHSWhat are the diagnostic criteria?HHSWhat monitoring is needed?HHSWhat if systolic BP is below 90?HHSHow do I know resolution?HypocalcaemiaWhat are the symptoms?HypocalcaemiaWhat are ECG signs?HypocalcaemiaWhat monitoring is required?HypocalcaemiaWhat if vitamin D deficient?HypokalaemiaAt what level does it require treatment?HypokalaemiaWhen can oral replacement be used?HypokalaemiaWhat are ECG signs?HypokalaemiaWhen should concentrated potassium be used?HyponatraemiaWhen is it considered severe?HyponatraemiaWhat investigations are required?HyponatraemiaHow to test for pseudohyponatraemia?HyponatraemiaHow is it treated?Infective endocarditisWhat antibiotics should be given?Infective endocarditisWhat blood tests are required?Infective endocarditisWhat imaging should be ordered?Infective endocarditisWho should this be discussed with?SAHHow is diagnosis confirmed?SAHWhat are the symptoms?SAHWhen should LP be performed?SAHHow should CT\-confirmed SAH be managed?PancreatitisHow is diagnosis confirmed?PancreatitisWhat tests should be ordered?PancreatitisWho should I refer to?PancreatitisWhen should I escalate to ICU?PericarditisWhat imaging should I request?PericarditisWhat ECG changes occur?PericarditisWhat is the treatment?PericarditisCan the patient be discharged?TachycardiaWhat investigations should I perform?TachycardiaWhat are differentials for broad complex tachycardia?TachycardiaHow do I escalate unstable tachycardia?TachycardiaWhat electrolyte targets should I aim for?Ureteric colicWhat tests should I perform?Ureteric colicWhat is the treatment?Ureteric colicWhen is discharge safe?Ureteric colicWhen should I refer to urology?Upper GI bleedHow do I assess severity?Upper GI bleedWhen to activate major haemorrhage protocol?Upper GI bleedWhat additional blood products are needed?Upper GI bleedWhat is the dose of terlipressin?Table A2:Oncology guideline\-based clinical questions\.GuidelineQuestionHypersensitivityIrinotecan anticholinergic syndromeHypersensitivityDrugs given in grade 2 hypersensitivity reactionsHypersensitivityHow to restart chemotherapy after a reactionHypersensitivityOxaliplatin hypersensitivity reactionDiarrhea \(SACT\)Are there contraindications to DOAC?Diarrhea \(SACT\)Can DOAC be used in severe renal impairment?Diarrhea \(SACT\)Do I need monitoring on DOAC?Diarrhea \(SACT\)How long to treat cancer\-associated VTE?Diarrhea \(SACT\)What if intolerant to DOACs?Diarrhea \(SACT\)Anticoagulation in BMI 135 kgDiarrhea \(SACT\)PICC\-associated thrombus managementDiarrhea \(SACT\)When to offer IVC filterDiarrhea \(SACT\)Apixaban dosing in cancer VTEDiarrhea \(SACT\)VTE with low plateletsDiarrhea \(SACT\)Octreotide administrationDiarrhea \(SACT\)Immunotherapy\-related diarrhea managementDiarrhea \(SACT\)Capecitabine overdose managementDiarrhea \(SACT\)Stool cultures before loperamide?Diarrhea \(SACT\)Treatment for severe diarrheaDiarrhea \(SACT\)Investigations for SACT diarrheaMSCCCan patient be mobilised?MSCCHow to contact coordinator?MSCCManagement after MRI confirmationMSCCDischarge if MRI negative?MSCCLMWH prophylaxis?MSCCSteroid initiation?MSCCClinical signs of MSCCMSCCInvestigations for unknown primaryMSCCFirst\-line investigationMSCCWho to contact locallyMagnesium disordersContraindicated drugsMagnesium disordersNeed for ECGMagnesium disordersManaging oral magnesium side effectsMagnesium disordersAdmission threshold Mg 0\.45Magnesium disordersSymptomsMagnesium disordersCausative drugsMagnesium disordersOral supplementationMagnesium disordersDefinition of severe hypomagnesaemiaMagnesium disordersIV monitoringMagnesium disordersOther electrolytesNeutropenic sepsisInfection with normal neutrophils?Neutropenic sepsisCentral line useNeutropenic sepsisNeed for chest X\-rayNeutropenic sepsisDiagnostic criteriaNeutropenic sepsisInitial managementNeutropenic sepsisLeg swelling post\-chemoNeutropenic sepsisAntibioticsNeutropenic sepsisEscalation after failureNeutropenic sepsisRequired investigationsNeutropenic sepsisNeutrophil threshold
## Prompt Listings

Listing 1:Baseline Prompt: Section Subpart ExtractionCreatesmallerclinicallycoherentsubpartsfromtheparagraphandidentifyrelevantquestions\.

PARAGRAPH\(verbatim\):

\-\-\-

\{paragraph\}

\-\-\-

Task:

\-Identify2\-6subpartsthatcaneachansweradistinctclinicalquestion\.

\-Subpartsmayoverlapandmaybedifferentlengths\.

\-Foreachsubpart:

\-Provideashorttitle

\-ProvideONEshortquestion\(<=16words\)

\-Providethesupporting’text’asaverbatimspancopiedEXACTLYfromtheparagraph\.

Rules:

\-Copyparagraphtextexactly;donotmodifywording\.

\-DoNOTinventcontentnotpresentintheparagraph\.

\-JSONONLY\.Nomarkdownorcommentary\.

ReturnSTRICTJSONwiththisschema:

\{

"subparts":\[

\{

"section\_id":"P1\-S1",

"title":"string",

"question":"<=16words;actionable",

"text":"verbatimspancopiedEXACTLYfromtheparagraph"

\}

\]

\}

### Topic Selection \(TS\)

Listing 2:Topic Selection Prompt \(TS\)Youareselectingthemostrelevantclinicalguidelinetopic\.

USERQUESTION:

\{user\_question\}

AVAILABLEGUIDELINETOPICIDs:

\{guideline\_ids\}

INSTRUCTIONS:

1\.SelectexactlyONEguideline\_idfromthelistabove\.

2\.Choosethemostdirectlyrelevanttopic\.

3\.Ifnoperfectmatchexists,selecttheclosestrelevanttopic\.

4\.DoNOTinventnewIDs\.

5\.DoNOToutputexplanationsoutsideJSON\.

OUTPUTFORMAT\(STRICTJSONONLY\):

\{

"guideline\_id":"<oneofthelistedIDs\>",

"rationale":"<max1shortsentence\>"

\}

### Line Selection Strategy \(LINES\)

Listing 3:Line Selection Prompt \(LINES\)Youwillanswertheuser’squestionbyselectingthemostrelevant

linesfromtheguideline\.

GOAL:

\-SelecttheMAXIMUMnumberofguidelinelinesthatarerelevant\.

\-Whenunsurewhetheralineisrelevant,INCLUDEitratherthanomitit\.

GUIDELINE\(withlinenumbers\):

\{numbered\_guideline\_text\}

Userquestion:

\{user\_question\}

Instructions:

\-AttheVERYENDofyourresponse,outputONLYlinenumbers

\(comma\-separatedorrangeslike12\-18\)\.

\-DoNOTincludeanyothertextafterthefinallinenumbers\.

### Verbatim Sentence Extraction \(COPY\)

Listing 4:Verbatim Sentence Extraction Prompt \(COPY\)Copythemostrelevantlinesfromthetexttoanswertheuser’squestion\.

GUIDELINE:

\{guideline\_sentences\}

Userquestion:

\{user\_question\}

Rules:

\-CopyONLYfromtheprovidedTEXT\.

\-Copyverbatim\(exactwording\)\.

\-Prefertheminimumsetoflinesthatfullyanswersthequestion\.

\-ReturnJSONonly\.

Schema:

\{"copied":"verbatimlinescopiedfromtheprovidedtext"\}

### JuniorDoc Strategy

#### Agent 1: Draft Clinical Answer

Listing 5:JuniorDoc Prompt: Agent 1 \(Draft Clinical Answer\)YouareAgent1,ajuniordoctoransweringfrommemory\.

ProvideyourbestapproximateanswerWITHOUTreading

anyexternaldocuments\.

Beconciseandclinic\-facing\.

Extractactionabledetailswheneverremembered

\(e\.g\.thresholds,doses,durations,monitoringsteps\)\.

#### Agent 2: Guideline Evidence Extraction

Listing 6:JuniorDoc Prompt: Agent 2 \(Guideline Evidence Extraction\)YouareAgent2\.

Inputs:

\-userquestion

\-guidelinetext

\-Agent1answer\(forcomparisononly\)

Task:

SelecttheMAXIMUMsubsetofFULLSENTENCEScopied

VERBATIMfromtheguidelinethateither:

\-SUPPORTAgent1

\-ADDdetailtoAgent1

\-CORRECTAgent1

Rules:

\-Copyonlyfromguidelinesentences

\-Preserveorder

\-ReturnJSONonly

\{"subset":\["sentence",\.\.\.\]\}

### Minimal Safe Extraction \(Safety\)

Listing 7:Minimal Safe Extraction Prompt \(Safety\)SelecttheSMALLESTorderedsubsetofFULLsentences

fromtheguidelinethatanswersthequestion\.

Eachsentencemusthaveonelabel:

answers\_question

adds\_required\_detail

safety\_critical

Rules:

\-Copyonlyfromguidelinesentences

\-Preserveorder

\-Noparaphrasing

\-ReturnJSON

\{

"sentences":\[

\{"text":"sentence","reason":"answers\_question"\}

\]

\}

Table A3:Representative questions across datasets\.Table A4:Examples of extraction behaviour across prompting strategies \(UCLH, gpt\-oss:120B\)\. The table compares reference answers with outputs generated by the LINES, COPY, and Safety strategies\. LINES reconstructs answers from selected guideline lines and consistently achieves high coverage of reference content\. COPY copies verbatim sentences from the guideline\. Safety selects the minimal subset of sentences required to answer the question, produces concise responses but may omit important details\. Original text is paraphrased\.Table A5:JuniorDoc pipeline examples \(UCLH, gpt\-oss:120B\)\. The table illustrates how intermediate steps transform the original uninformed LLM answer\. Comparison with reference answers highlights presence of non\-essential content\. Original text is paraphrased\.Table A6:Automatic evaluation of prompting strategies for GPT\-OSS\-120B with Somerset NHS guideline sections selected up to a cumulative similarity mass threshold \(0\.95\) using the text\+question representations\. Under this setting,LINESachieves consistently higher recall than other prompting methods \(e\.g\., word recall≈\\approx0\.65\), with statistically significant improvements overCOPY, while maintaining moderate answer length\. Multi\-agent \(JuniorDoc\) further increases recall marginally but at the cost of longer outputs\. Bold indicates the best\-performing method per metric, and second\-best values are underlined\.MethodLenR\-LPRF1Term PTerm RTerm F1copy42\.500\.43210\.50430\.51860\.46420\.50340\.51250\.4706lines117\.280\.3406\*0\.30060\.6539\*0\.3639\*0\.3755\*0\.6291\*0\.4191\*junior142\.000\.2549\*0\.21330\.6543\*0\.2828\*0\.2567\*0\.6372\*0\.3228\*safety38\.620\.3774\*0\.46610\.4596\*0\.4089\*0\.47860\.47420\.4274\*Table A7:Automatic evaluation of prompting strategies across model families with retrieval constrained to NICE guideline sections selected up to a cumulative similarity mass threshold \(0\.95\)\. Metrics include ROUGE\-L \(R\-L, sentence\-level recall\), word\-level precision \(P\), recall \(R\), and F1, as well as medical term precision, recall, and F1\. Answer length \(Len\) is reported as mean word count \(lower is better\)\. Statistical significance is assessed via paired bootstrap resampling \(10,000 draws with replacement\), with 95% confidence intervals and two\-sided p\-values computed from the bootstrap distribution; significance \(\*\) indicatesp<0\.05p<0\.05relative to theCOPYbaseline\. Constraining retrieval substantially reduces input context \(from∼\\sim5,000 words to∼\\sim835 words\)\.LINESconsistently achieves the highest recall among prompting methods \(e\.g\., word recall of∼\\sim0\.71 across models\), with statistically significant improvements overCOPY\. Bold indicates the best\-performing method per metric, and second\-best values are underlined\.MethodLenR\-LPRF1Term PTerm RTerm F1DeepSeek\-V3copy65\.500\.63600\.64110\.86340\.69440\.65910\.59120\.5857lines222\.760\.3446\*0\.27760\.88830\.3735\*0\.47560\.7293\*0\.5168\*junior147\.190\.3443\*0\.24700\.8963\*0\.3609\*0\.49100\.6655\*0\.5084\*safety77\.730\.4979\*0\.42070\.85780\.5234\*0\.5453\*0\.6370\*0\.5407\*GPT\-5\-minicopy68\.660\.59640\.58560\.89110\.65580\.60620\.66040\.5819lines440\.750\.2111\*0\.14720\.9433\*0\.2279\*0\.45840\.7209\*0\.4974\*junior193\.010\.3199\*0\.23170\.9155\*0\.3340\*0\.47290\.6958\*0\.5054\*safety91\.160\.4651\*0\.36820\.87740\.4848\*0\.57030\.63290\.5520Qwen3\-235Bcopy46\.340\.62960\.65040\.82500\.69380\.63950\.63200\.5967lines225\.030\.3442\*0\.29190\.85240\.3850\*0\.43970\.7222\*0\.4878\*junior96\.140\.4840\*0\.41870\.9080\*0\.5228\*0\.5381\*0\.6747\*0\.5446\*safety65\.880\.4976\*0\.45930\.80290\.5438\*0\.59060\.59190\.5432\*Table A8:Examples illustrating minor addition and minor omission in LLM outputs, with clinical interpretation\. All examples are paraphrased\.

## References

- \[1\]F\. W\. Arnold, A\. S\. LaJoie, G\. N\. Brock, P\. Peyrani, J\. Rello, R\. Menéndez, G\. Lopardo, A\. Torres, P\. Rossi, J\. A\. Ramirez,et al\.\(2009\)Improving outcomes in elderly patients with community\-acquired pneumonia by adhering to national guidelines: community\-acquired pneumonia organization international cohort study results\.Archives of internal medicine169\(16\),pp\. 1515–1524\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p1.1)\.
- \[2\]Q\. Chen, Y\. Hu, X\. Peng, Q\. Xie, Q\. Jin, A\. Gilson, M\. B\. Singer, X\. Ai, P\. Lai, Z\. Wang, V\. K\. Keloth, K\. Raja, J\. Huang, H\. He, F\. Lin, J\. Du, R\. Zhang, W\. J\. Zheng, R\. A\. Adelman, Z\. Lu, and H\. Xu\(2025\-04\)Benchmarking large language models for biomedical natural language processing applications and recommendations\.Nat\. Commun\.16\(1\),pp\. 3280\(en\)\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p3.1)\.
- \[3\]V\. Coba, M\. Whitmill, R\. Mooney, H\. M\. Horst, M\. Brandt, B\. Digiovine, M\. Mlynarek, B\. McLellan, G\. Boleski, J\. Yang,et al\.\(2011\)Resuscitation bundle compliance in severe sepsis and septic shock: improves survival, is better late than never\.Journal of intensive care medicine26\(5\),pp\. 304–313\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p1.1)\.
- \[4\]J\. Daniels, E\. Robinson, E\. Jenkinson, and E\. Carlton\(2024\)Perceived barriers and opportunities to improve working conditions and staff retention in emergency departments: a qualitative study\.Emergency Medicine Journal41\(4\),pp\. 257–265\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p2.1)\.
- \[5\]N\. C\. Dean, M\. P\. Silver, K\. A\. Bateman, B\. James, C\. J\. Hadlock, and D\. Hale\(2001\)Decreased mortality after implementation of a treatment guideline for community\-acquired pneumonia\.The American journal of medicine110\(6\),pp\. 451–457\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p1.1)\.
- \[6\]Q\. Ding, E\. H\. Q\. Zhang, F\. Jozsa, and J\. Ive\(2025\-11\)Building a silver\-standard dataset from NICE guidelines for clinical LLMs\.arXiv \[cs\.CL\]\.Cited by:[Data](https://arxiv.org/html/2606.12897#Sx2.SSx2.p4.1)\.
- \[7\]S\. Freedman, E\. Golberstein, T\. Huang, D\. J\. Satin, and L\. B\. Smith\(2021\)Docs with their eyes on the clock? the effect of time pressures on primary care productivity\.Journal of health economics77,pp\. 102442\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p2.1)\.
- \[8\]Gemma Team, T\. Mesnard, C\. Hardin, R\. Dadashi, S\. Bhupatiraju, S\. Pathak, L\. Sifre, M\. Rivière, M\. S\. Kale, J\. Love, P\. Tafti, L\. Hussenot, P\. G\. Sessa, A\. Chowdhery, A\. Roberts, A\. Barua, A\. Botev, A\. Castro\-Ros, A\. Slone, A\. Héliou, A\. Tacchetti, A\. Bulanova, A\. Paterson, B\. Tsai, B\. Shahriari, C\. L\. Lan, C\. A\. Choquette\-Choo, C\. Crepy, D\. Cer, D\. Ippolito, D\. Reid, E\. Buchatskaya, E\. Ni, E\. Noland, G\. Yan, G\. Tucker, G\. Muraru, G\. Rozhdestvenskiy, H\. Michalewski, I\. Tenney, I\. Grishchenko, J\. Austin, J\. Keeling, J\. Labanowski, J\. Lespiau, J\. Stanway, J\. Brennan, J\. Chen, J\. Ferret, J\. Chiu, J\. Mao\-Jones, K\. Lee, K\. Yu, K\. Millican, L\. L\. Sjoesund, L\. Lee, L\. Dixon, M\. Reid, M\. Mikuła, M\. Wirth, M\. Sharman, N\. Chinaev, N\. Thain, O\. Bachem, O\. Chang, O\. Wahltinez, P\. Bailey, P\. Michel, P\. Yotov, R\. Chaabouni, R\. Comanescu, R\. Jana, R\. Anil, R\. McIlroy, R\. Liu, R\. Mullins, S\. L\. Smith, S\. Borgeaud, S\. Girgin, S\. Douglas, S\. Pandya, S\. Shakeri, S\. De, T\. Klimenko, T\. Hennigan, V\. Feinberg, W\. Stokowiec, Y\. Chen, Z\. Ahmed, Z\. Gong, T\. Warkentin, L\. Peran, M\. Giang, C\. Farabet, O\. Vinyals, J\. Dean, K\. Kavukcuoglu, D\. Hassabis, Z\. Ghahramani, D\. Eck, J\. Barral, F\. Pereira, E\. Collins, A\. Joulin, N\. Fiedel, E\. Senter, A\. Andreev, and K\. Kenealy\(2024\-03\)Gemma: open models based on gemini research and technology\.arXiv \[cs\.CL\]\.Cited by:[LLM Types](https://arxiv.org/html/2606.12897#Sx2.SSx4.p2.1)\.
- \[9\]E\. Guerra\-Farfan, Y\. Garcia\-Sanchez, M\. Jornet\-Gibert, J\. H\. Nuñez, M\. Balaguer\-Castro, and K\. Madden\(2023\)Clinical practice guidelines: the good, the bad, and the ugly\.Injury54,pp\. S26–S29\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p1.1)\.
- \[10\]B\. H\. Kann, S\. B\. Johnson, H\. J\. Aerts, R\. H\. Mak, and P\. L\. Nguyen\(2020\)Changes in length and complexity of clinical practice guidelines in oncology, 1996\-2019\.JAMA Network Open3\(3\),pp\. e200841–e200841\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p2.1)\.
- \[11\]Z\. Kraljevic, T\. Searle, A\. Shek, L\. Roguski, K\. Noor, D\. Bean, A\. Mascio, L\. Zhu, A\. A\. Folarin, A\. Roberts, R\. Bendayan, M\. P\. Richardson, R\. Stewart, A\. D\. Shah, W\. K\. Wong, Z\. Ibrahim, J\. T\. Teo, and R\. J\. B\. Dobson\(2021\-07\)Multi\-domain clinical natural language processing with MedCAT: the medical concept annotation toolkit\.Artif\. Intell\. Med\.117,pp\. 102083\.External Links:ISSN 0933\-3657,[Document](https://dx.doi.org/10.1016/j.artmed.2021.102083)Cited by:[Automatic Evaluation](https://arxiv.org/html/2606.12897#Sx2.SSx5.SSS0.Px1.p5.1)\.
- \[12\]S\. Kresevic, M\. Giuffrè, M\. Ajcevic, A\. Accardo, L\. S\. Crocè, and D\. L\. Shung\(2024\-04\)Optimization of hepatological clinical guidelines interpretation by large language models: a retrieval augmented generation\-based framework\.NPJ Digit\. Med\.7\(1\),pp\. 102\(en\)\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p1.1),[Introduction](https://arxiv.org/html/2606.12897#Sx1.p3.1),[Introduction](https://arxiv.org/html/2606.12897#Sx1.p5.1)\.
- \[13\]M\. Lewis, S\. Thio, A\. Roberts, C\. Siju, W\. Mukit, R\. Kuruvilla, Z\. J\. Jiang, N\. Möller\-Grell, A\. Borakati, R\. J\. B\. Dobson, and S\. Denaxas\(2025\-12\)Grounding large language models in clinical evidence: a retrieval\-augmented generation system for querying UK NICE clinical guidelines\.arXiv \[cs\.CL\]\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p3.1),[Introduction](https://arxiv.org/html/2606.12897#Sx1.p5.1)\.
- \[14\]P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela\(2020\-05\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.arXiv \[cs\.CL\]\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p3.1)\.
- \[15\]C\. Lin\(2004\)ROUGE: a package for automatic evaluation of summaries\.InText Summarization Branches Out,Cited by:[Automatic Evaluation](https://arxiv.org/html/2606.12897#Sx2.SSx5.SSS0.Px1.p2.1)\.
- \[16\]A\. Lv, R\. Xie, X\. Sun, Z\. Kang, and R\. Yan\(2024\-09\)Language models “grok” to copy\.arXiv \[cs\.CL\]\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p3.1)\.
- \[17\]G\. Monti, E\. Rezoagli, A\. Calini, A\. Nova, S\. Marchesi, G\. Nattino, G\. Carrara, S\. Morra, F\. Cortellaro, M\. Savioli,et al\.\(2023\)Effect of a quality improvement program on compliance to the sepsis bundle in non\-icu patients: a multicenter prospective before and after cohort study\.Frontiers in Medicine10,pp\. 1215341\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p1.1)\.
- \[18\]K\. K\. Y\. Ng, I\. Matsuba, and P\. C\. Zhang\(2025\-01\)RAG in health care: a novel framework for improving communication and decision\-making by addressing LLM limitations\.NEJM AI2\(1\) \(en\)\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p3.1)\.
- \[19\]OpenAI, S\. Agarwal, L\. Ahmad, J\. Ai, S\. Altman, A\. Applebaum, E\. Arbus, R\. K\. Arora, Y\. Bai, B\. Baker, H\. Bao, B\. Barak, A\. Bennett, T\. Bertao, N\. Brett, E\. Brevdo, G\. Brockman, S\. Bubeck, C\. Chang, K\. Chen, M\. Chen, E\. Cheung, A\. Clark, D\. Cook, M\. Dukhan, C\. Dvorak, K\. Fives, V\. Fomenko, T\. Garipov, K\. Georgiev, M\. Glaese, T\. Gogineni, A\. Goucher, L\. Gross, K\. G\. Guzman, J\. Hallman, J\. Hehir, J\. Heidecke, A\. Helyar, H\. Hu, R\. Huet, J\. Huh, S\. Jain, Z\. Johnson, C\. Koch, I\. Kofman, D\. Kundel, J\. Kwon, V\. Kyrylov, E\. Y\. Le, G\. Leclerc, J\. P\. Lennon, S\. Lessans, M\. Lezcano\-Casado, Y\. Li, Z\. Li, J\. Lin, J\. Liss, Lily, Liu, J\. Liu, K\. Lu, C\. Lu, Z\. Martinovic, L\. McCallum, J\. McGrath, S\. McKinney, A\. McLaughlin, S\. Mei, S\. Mostovoy, T\. Mu, G\. Myles, A\. Neitz, A\. Nichol, J\. Pachocki, A\. Paino, D\. Palmie, A\. Pantuliano, G\. Parascandolo, J\. Park, L\. Pathak, C\. Paz, L\. Peran, D\. Pimenov, M\. Pokrass, E\. Proehl, H\. Qiu, G\. Raila, F\. Raso, H\. Ren, K\. Richardson, D\. Robinson, B\. Rotsted, H\. Salman, S\. Sanjeev, M\. Schwarzer, D\. Sculley, H\. Sikchi, K\. Simon, K\. Singhal, Y\. Song, D\. Stuckey, Z\. Sun, P\. Tillet, S\. Toizer, F\. Tsimpourlas, N\. Vyas, E\. Wallace, X\. Wang, M\. Wang, O\. Watkins, K\. Weil, A\. Wendling, K\. Whinnery, C\. Whitney, H\. Wong, L\. Yang, Y\. Yang, M\. Yasunaga, K\. Ying, W\. Zaremba, W\. Zhan, C\. Zhang, B\. Zhang, E\. Zhang, and S\. Zhao\(2025\-08\)Gpt\-oss\-120b & gpt\-oss\-20b model card\.arXiv \[cs\.CL\]\.Cited by:[LLM Types](https://arxiv.org/html/2606.12897#Sx2.SSx4.p1.1)\.
- \[20\]A\. Pal, L\. K\. Umapathi, and M\. Sankarasubbu\(2023\-12\)Med\-HALT: medical domain hallucination test for large language models\.InProceedings of the 27th Conference on Computational Natural Language Learning \(CoNLL\),Stroudsburg, PA, USA,pp\. 314–334\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p3.1)\.
- \[21\]B\. Qumseya, A\. Goddard, A\. Qumseya, D\. Estores, P\. V\. Draganov, and C\. Forsmark\(2021\)Barriers to clinical practice guideline implementation among physicians: a physician survey\.International Journal of General Medicine,pp\. 7591–7598\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p2.1)\.
- \[22\]E\. Scraggs, L\. Brereton, J\. Newbould, S\. Drabble, C\. Tiefensee, D\. Schweppenstedde, C\. Miani, and T\. Ling\(2012\)Factors that encourage or discourage doctors from acting in accordance with good practice final report prepared for the general medical council\.Technical reportCited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p1.1)\.
- \[23\]K\. Singhal, T\. Tu, J\. Gottweis, R\. Sayres, E\. Wulczyn, M\. Amin, L\. Hou, K\. Clark, S\. R\. Pfohl, H\. Cole\-Lewis, D\. Neal, Q\. M\. Rashid, M\. Schaekermann, A\. Wang, D\. Dash, J\. H\. Chen, N\. H\. Shah, S\. Lachgar, P\. A\. Mansfield, S\. Prakash, B\. Green, E\. Dominowska, B\. Agüera Y Arcas, N\. Tomašev, Y\. Liu, R\. Wong, C\. Semturs, S\. S\. Mahdavi, J\. K\. Barral, D\. R\. Webster, G\. S\. Corrado, Y\. Matias, S\. Azizi, A\. Karthikesalingam, and V\. Natarajan\(2025\-03\)Toward expert\-level medical question answering with large language models\.Nat\. Med\.31\(3\),pp\. 943–950\(en\)\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p3.1)\.
- \[24\]I\. Torjesen\(2021\)Doctors are under more work pressure than during height of covid\-19 pandemic in 2020\.BMJ: British Medical Journal \(Online\)375\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p2.1)\.
- \[25\]E\. Tsiga, E\. Panagopoulou, N\. Sevdalis, A\. Montgomery, and A\. Benos\(2013\)The influence of time pressure on adherence to guidelines in primary care: an experimental study\.BMJ open3\(4\),pp\. e002700\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p2.1)\.
- \[26\]A\. Van der Vegt, G\. Zuccon, B\. Koopman, and A\. Deacon\(2020\)How searching under time pressure impacts clinical decision making\.Journal of the Medical Library Association: JMLA108\(4\),pp\. 564\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p2.1)\.
- \[27\]L\. B\. VanWagner, J\. L\. Holl, S\. Montag, D\. Gregory, S\. Connolly, M\. Kosirog, P\. Campbell, S\. Pine, A\. Daud, D\. Finn,et al\.\(2020\)Blood pressure control according to clinical practice guidelines is associated with decreased mortality and cardiovascular events among liver transplant recipients\.American Journal of Transplantation20\(3\),pp\. 797–807\.Cited by:[Introduction](https://arxiv.org/html/2606.12897#Sx1.p1.1)\.
- \[28\]A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. Qiu\(2025\-05\)Qwen3 technical report\.arXiv \[cs\.CL\]\.Cited by:[LLM Types](https://arxiv.org/html/2606.12897#Sx2.SSx4.p1.1)\.