Information-seeking failures of large language models in agentic clinical reasoning
Summary
The paper develops an agentic evaluation framework for clinical reasoning in hematologic oncology, finding that LLMs primarily fail due to systematic information-seeking deficits rather than insufficient knowledge, with error patterns resembling cognitive biases in novice clinicians.
View Cached Full Text
Cached at: 07/14/26, 04:19 AM
# Information-seeking failures of large language models in agentic clinical reasoning Source: [https://arxiv.org/abs/2607.10275](https://arxiv.org/abs/2607.10275) Authors:[Krischan Braitsch](https://arxiv.org/search/cs?searchtype=author&query=Braitsch,+K),[Laura K\. Schmalbrock](https://arxiv.org/search/cs?searchtype=author&query=Schmalbrock,+L+K),[Theresa Weltermann](https://arxiv.org/search/cs?searchtype=author&query=Weltermann,+T),[Andrew F\. Berdel](https://arxiv.org/search/cs?searchtype=author&query=Berdel,+A+F),[Isabella Miller](https://arxiv.org/search/cs?searchtype=author&query=Miller,+I),[Kai Tran](https://arxiv.org/search/cs?searchtype=author&query=Tran,+K),[Michael Heider](https://arxiv.org/search/cs?searchtype=author&query=Heider,+M),[Sabrina Kraus](https://arxiv.org/search/cs?searchtype=author&query=Kraus,+S),[Florian Bassermann](https://arxiv.org/search/cs?searchtype=author&query=Bassermann,+F),[Jacqueline Lammert](https://arxiv.org/search/cs?searchtype=author&query=Lammert,+J),[Sebastian Ziegelmayer](https://arxiv.org/search/cs?searchtype=author&query=Ziegelmayer,+S),[Marcus Makowski](https://arxiv.org/search/cs?searchtype=author&query=Makowski,+M),[Lisa C\. Adams](https://arxiv.org/search/cs?searchtype=author&query=Adams,+L+C),[Keno K\. Bressem](https://arxiv.org/search/cs?searchtype=author&query=Bressem,+K+K) [View PDF](https://arxiv.org/pdf/2607.10275) > Abstract:Large language models achieve high scores on medical knowledge assessments, yet clinical reasoning requires actively deciding what to investigate under uncertainty\. We developed an agentic evaluation framework in hematologic oncology in which models must proactively request clinical data across three sequential rounds before committing to a diagnosis and treatment plan\. Across 32 frontier models, the best achieved only 68% overall accuracy\. Information utilization, the fraction of available data actually requested, was the strongest predictor of diagnostic accuracy \(R = 0\.69, P < 0\.001\), yet utilization collapsed from 57% to 26% in the final round, leaving molecular and cytogenetic data critical for treatment selection unexamined\. Reasoning traces scored high on a clinical reasoning rubric \(91% above threshold\) but decorrelated from accuracy, revealing a gap between locally coherent rationales and globally correct conclusions\. Error analysis identified search satisficing, anchoring and premature closure as the dominant failure modes, the same cognitive biases that characterize novice clinicians under dual\-process models of diagnostic reasoning\. These findings demonstrate that the primary limitation of current models in clinical oncology is not insufficient medical knowledge but a systematic failure of information\-seeking under uncertainty\. ## Submission history From: Keno Bressem \[[view email](https://arxiv.org/show-email/0c212e4b/2607.10275)\] **\[v1\]**Sat, 11 Jul 2026 12:10:33 UTC \(2,783 KB\)
Similar Articles
Large Language Models Show Metacognitive Sensitivity in Medical Reasoning
A controlled study evaluates the metacognitive sensitivity of large language models in medical reasoning, finding partial but flawed confidence calibration that varies with evidence strength and conflicting scenarios.
Reasoning in Real World Clinical Care: Why Large Language Models Are Not Yet Safe for Autonomous Clinical Decision Support
This Perspective paper argues that large language models are not yet safe for autonomous clinical decision support, particularly in triage of undifferentiated patients, due to lack of robust evaluation under incomplete information and asymmetric costs of missed diagnoses.
Reinforcement Learning for Evidence-Seeking Diagnostic Reasoning with Large Language Models
This paper proposes a reinforcement learning framework for evidence-seeking diagnostic reasoning using LLMs. The RL-trained 7B model outperforms larger models in multilingual clinical consultation tasks, showing that specialized RL can distill high-level clinical reasoning.
AIPatient Arena: EHR-grounded evaluation of large language models in end-to-end clinical consultation workflows
Introduces AIPatient Arena, an EHR-grounded evaluation framework for assessing LLMs across multiple dimensions of clinical competence. The study reveals strengths in interviewing and ethics but weaknesses in handling ambiguity and diagnostic accuracy.
LLM Doesn't Know What It Doesn't Know: Detecting Epistemic Blind Spots via Cross-Model Attribution Divergence on Clinical Tabular Data
This paper explores Large Language Models' inability to recognize their knowledge limits on structured clinical data, proposing a cross-model attribution divergence method to detect epistemic blind spots. The approach improves calibration and accuracy without training by combining few-shot examples and SHAP-derived feature evidence.