Human-LLM Dialogue Improves Diagnostic Accuracy in Emergency Care
Summary
This study evaluates how interactive dialogue with an LLM (via the MedSyn system) improves diagnostic accuracy for physicians in emergency care settings, showing significant gains for residents on difficult cases.
View Cached Full Text
Cached at: 05/12/26, 07:16 AM
# Human-LLM Dialogue Improves Diagnostic Accuracy in Emergency Care
Source: [https://arxiv.org/abs/2605.08533](https://arxiv.org/abs/2605.08533)
Authors:[Burcu Sayin](https://arxiv.org/search/cs?searchtype=author&query=Sayin,+B),[Ngoc Vo Hong](https://arxiv.org/search/cs?searchtype=author&query=Hong,+N+V),[Ipek Baris Schlicht](https://arxiv.org/search/cs?searchtype=author&query=Schlicht,+I+B),[Jacopo Staiano](https://arxiv.org/search/cs?searchtype=author&query=Staiano,+J),[Pasquale Minervini](https://arxiv.org/search/cs?searchtype=author&query=Minervini,+P),[Sara Allievi](https://arxiv.org/search/cs?searchtype=author&query=Allievi,+S),[Nicola Susca](https://arxiv.org/search/cs?searchtype=author&query=Susca,+N),[Nicola Osti](https://arxiv.org/search/cs?searchtype=author&query=Osti,+N),[Alberto Maino](https://arxiv.org/search/cs?searchtype=author&query=Maino,+A),[Vito Racanelli](https://arxiv.org/search/cs?searchtype=author&query=Racanelli,+V),[Andrea Passerini](https://arxiv.org/search/cs?searchtype=author&query=Passerini,+A)
[View PDF](https://arxiv.org/pdf/2605.08533)
> Abstract:Clinical decision\-making in emergency medicine demands rapid, accurate diagnoses under uncertainty\. Despite benchmark progress, evidence for LLMs as interactive aids in live physician workflows remains sparse\. MedSyn lets physicians iteratively query an LLM provided with the full clinical record while initially viewing only the chief complaint\. Seven physicians \(three seniors, four residents\) completed baseline and AI\-assisted sessions across 52 MIMIC\-IV cases stratified by difficulty\. Blinded evaluation showed residents' Hard\-case correctness rose from 0\.589 to 0\.734; difficulty\-standardised completely\-correct rates confirmed a medium effect \(\{\\Delta\} = 0\.092; p = 0\.071; d = 0\.47\)\. Automated metrics corroborated these gains: standardised any\-match accuracy improved by 0\.156 \(p < 0\.0001\), and residents showed the largest F1 gain \(\{\\Delta\} = 0\.138; p < 0\.0001\)\. Dialogue analysis revealed expertise\-dependent strategies \(seniors asked targeted, hypothesis\-driven questions; residents relied on broader queries\) and cross\-expertise concordance increased \(\{\\Delta\} = 0\.145; p < 0\.0001\)\. Interactive LLM support meaningfully enhances diagnostic reasoning\.
## Submission history
From: Burcu Sayin Günel \[[view email](https://arxiv.org/show-email/91bf1b8b/2605.08533)\] **\[v1\]**Fri, 8 May 2026 22:40:10 UTC \(1,666 KB\)Similar Articles
Are LLMs Ready to Assist Physicians? PhysAssistBench for Interactive Doctor-Patient-EHR Assistance
Introduces PhysAssistBench, a benchmark for evaluating LLMs in interactive doctor-patient-EHR assistance. Experiments show current models are unreliable in this setting, highlighting the need for coordinated capabilities.
Evaluating medical AI under missing information: same-provider judges and human raters change apparent safety
This paper extends missing-information stress-testing to open-ended medical conversation, finding that LLM judge choice materially changes apparent safety and that LLM judges are more permissive than clinicians.
Synthesis and Evaluation of Long-term History-aware Medical Dialogue
This paper introduces a framework for synthesizing long-term medical dialogue datasets using LLMs, and creates MediLongChat with three benchmark tasks to evaluate healthcare agents' memory and reasoning capabilities. Experiments show that even state-of-the-art LLMs struggle with these tasks.
Measuring Epistemic Resilience of LLMs Under Misleading Medical Context
Introduces MedMisBench to measure LLMs' ability to maintain correct medical reasoning under misleading context. Shows that accuracy drops sharply from 71.1% to 38.0% under adversarial conditions, with potential harm flagged by clinical panel.
MedAction: Towards Active Multi-turn Clinical Diagnostic LLMs
This paper introduces MedAction, a framework for training LLMs on active, multi-turn clinical diagnosis by simulating iterative test ordering and hypothesis updates. It presents a new dataset, MedAction-32K, and demonstrates state-of-the-art performance for open-source models on medical benchmarks.