Tag
A controlled study evaluates the metacognitive sensitivity of large language models in medical reasoning, finding partial but flawed confidence calibration that varies with evidence strength and conflicting scenarios.
Presents RESPClinBench, a real-world scenario benchmark for respiratory clinical decision-making, evaluating seven LLMs on COPD and pulmonary nodule cases. Finds task-specific limitations including imaging hallucination and medication-safety risks.
MEDSYN is a multilingual multimodal benchmark for evaluating MLLMs on complex clinical cases with up to 7 distinct visual evidence types per case. The study reveals that while frontier models match human experts on differential diagnosis generation, all MLLMs show significant gaps in final diagnosis selection due to poor synthesis of heterogeneous clinical evidence.