Clinician use of language models diverges from how the models are evaluated

Hugging Face Daily Papers Papers
clinical-ai llm-evaluation healthcare benchmarking human-ai-interaction natural-language-processing

Summary

This paper analyzes 127,833 real clinical queries sent by 6,342 healthcare providers to an institutional LLM assistant, finds that actual use (dominated by documentation and knowledge retrieval) diverges sharply from the task mix covered by 58 public clinical benchmarks, arguing that benchmark scores poorly predict real-world clinical AI performance.

Large language model (LLM) assistants are being deployed to clinicians across health systems, and judgments about their readiness rest largely on benchmark scores, most of them derived from examination questions or curated cases. A benchmark predicts performance in deployment only to the extent that its items resemble real use, yet whether benchmarks reflect the work these systems receive has rarely been measured. Here we analyze 127,833 queries sent by 6,342 physicians, advanced practice providers and nurses in 35 specialties to an institutional assistant during an eight-month roll-out. We characterize each query with RCQ-Map, a clinician-validated framework grounded in taxonomies of clinical questions and of LLM evaluation, which records its task, intent, answerability, missing information and potential harm. Documentation and administration (36.2%) and knowledge retrieval (28.9%) made up nearly two-thirds of use, and diagnosis 3.7%; more than a third of queries could not be answered well as posed. Applying RCQ-Map to 58 public benchmarks drawn from major evaluation suites and frontier model reports, which we assemble into the Clinical AI Benchmark Atlas, showed that the median benchmark contained no documentation requests and shared 31% of the task mix of real use, less than an even spread across task categories would. Benchmarks in suites designed to resemble clinical practice were individually no closer to real use than those used in frontier model reports. Benchmark scores therefore say little about how clinical AI performs on most of the work it is actually given, and evaluation should be matched to real clinical use.
Original Article
View Cached Full Text

Cached at: 10/12/26, 06:41 PM

Paper page - Clinician use of language models diverges from how the models are evaluated

Source: https://huggingface.co/papers/2610.11069 Published on Oct 8

·

Submitted byhttps://huggingface.co/alyakin314

Antonon Oct 12

Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Largelanguagemodel(LLM)assistantsarebeingdeployedtocliniciansacrosshealthsystems,andjudgmentsabouttheirreadinessrestlargelyonbenchmarkscores,mostofthemderivedfromexaminationquestionsorcuratedcases.Abenchmarkpredictsperformanceindeploymentonlytotheextentthatitsitemsresemblerealuse,yetwhetherbenchmarksreflecttheworkthesesystemsreceivehasrarelybeenmeasured.Hereweanalyze127,833queriessentby6,342physicians,advancedpracticeprovidersandnursesin35specialtiestoaninstitutionalassistantduringaneight-monthroll-out.WecharacterizeeachquerywithRCQ-Map,aclinician-validatedframeworkgroundedintaxonomiesofclinicalquestionsandofLLMevaluation,whichrecordsitstask,intent,answerability,missinginformationandpotentialharm.Documentationandadministration(36.2%)andknowledgeretrieval(28.9%)madeupnearlytwo-thirdsofuse,anddiagnosis3.7%;morethanathirdofqueriescouldnotbeansweredwellasposed.ApplyingRCQ-Mapto58publicbenchmarksdrawnfrommajorevaluationsuitesandfrontiermodelreports,whichweassembleintotheClinicalAIBenchmarkAtlas,showedthatthemedianbenchmarkcontainednodocumentationrequestsandshared31%ofthetaskmixofrealuse,lessthananevenspreadacrosstaskcategorieswould.Benchmarksinsuitesdesignedtoresembleclinicalpracticewereindividuallynoclosertorealusethanthoseusedinfrontiermodelreports.BenchmarkscoresthereforesaylittleabouthowclinicalAIperformsonmostoftheworkitisactuallygiven,andevaluationshouldbematchedtorealclinicaluse.

View arXiv pageView PDFGitHub2Add to collection

Get this paper in your agent:

hf papers read 2610\.11069

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.11069 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.11069 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.11069 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy

arXiv cs.CL

KlinikeBench introduces a clinician-verified benchmark of 333 interactive clinical tasks where LMs must gather patient history, use clinical tools, and reach a diagnosis within a turn budget. Across 31 models, the best (e.g., GPT-6-astra, Claude Opus 5) succeed on under 30% of tasks despite 90.7% diagnostic accuracy, revealing a large gap between diagnostic knowledge and real-world clinical interaction.