Tag
TreeProbe is the first cultural-bias benchmark for Tibetan medicine in LLMs, containing 4,719 expert-adjudicated items across 467 diseases and 10 subtasks, revealing systematic external ontology drift in current models.
EarlyDx is a new large-scale benchmark for evaluating LLMs on open-ended, evidence-supported diagnosis generation at emergency department admission, built from 154,834 MIMIC-IV encounters. It reveals that even frontier and medical-specialized models struggle to synthesize admission-time evidence, with post-training only partially improving inference-dependent recall.
MyoCardBench is a real-world benchmark for evaluating large language models in cardiovascular care, comprising 2,263 items across 13 tasks. GPT-5.4 achieved the highest overall score, demonstrating strengths in full-cycle care and multimodal interpretation.
Proposes a Collaborative Meta Knowledge Enhancement (COME) framework for dementia etiology diagnosis that injects heterogeneity-aware embeddings into a unified Transformer architecture, achieving state-of-the-art performance across multiple independent cohorts.
This paper presents a retrieval-augmented, multi-agent LLM framework with human-in-the-loop for detecting cutaneous immune-related adverse events from clinical notes, achieving higher accuracy, improved inter-rater agreement, and halved review time compared to manual review.
This paper shows that Monte Carlo dropout provides epistemic uncertainty signals for chest radiograph classifiers, which improves error detection and reduces confident misdiagnoses in clinical decision-support agents when communicated as a binary error-risk flag.
OpenAI is rolling out ChatGPT Health to all US users, allowing them to connect medical records and health-tracking data, with claims of clinician-level reasoning and integration with GPT-5.6 Sol.
This paper extends missing-information stress-testing to open-ended medical conversation, finding that LLM judge choice materially changes apparent safety and that LLM judges are more permissive than clinicians.
This paper describes TalTech's systems for generating SOAP notes directly from doctor-patient conversation audio, using Voxtral models fine-tuned with supervised learning and DAPO reinforcement learning. Their submissions ranked first in both tracks of the BeTraC challenge, achieving high concept accuracy and low hallucination rates.
Tri-Net v2 is an open-source implementation of a Scientific Reports paper for unified skin lesion and symptom-based monkeypox detection.
Kimi K3 AI model successfully reads a chest X-ray after OpenMed removes all 23 patient identifiers from the DICOM data, ensuring privacy. The model correctly identifies a left-sided whiteout.
Thinking Machines' Inkling, a 975B parameter model running locally on a Mac Studio via llama.cpp, listened to a doctor's visit audio and accurately diagnosed heart failure from subtle cues in small talk, demonstrating advanced medical reasoning without leaving the machine.
@MaziyarPanahi runs GLM-5.2 and Bonsai 27B locally on a Mac Studio using llama.cpp to process a 3-year patient chart, catching a dangerous drug interaction that was previously flagged but overlooked. The models operate entirely on-device under Apache-2.0, with Bonsai answering queries in ~2s and @PrismML claiming a 1-bit build fits an iPhone.
Evaluates agentic LLM systems for generating breast cancer treatment recommendations using 72 clinical cases, finding that the best system (Claude Opus 4.8 with D&C+SA pipeline) achieved a global score of 0.594 but remains insufficient for unsupervised clinical use due to persistent errors.
The paper develops an agentic evaluation framework for clinical reasoning in hematologic oncology, finding that LLMs primarily fail due to systematic information-seeking deficits rather than insufficient knowledge, with error patterns resembling cognitive biases in novice clinicians.
This paper proposes a framework that uses the Toulmin model of argumentation to structure ML-based retinal diagnosis from OCT images, integrating biomarker extraction, medical LLM reasoning (MedGemma), and similarity measures (MedSigLip) for interpretable and evidence-based diagnostic assistance.
Recommend Ant Ling large model API, which gives away 1 million tokens daily, excels in healthcare, supports OpenAI SDK compatible integration, and provides quick start documentation.
This paper presents HCC-STAR, a clinically aligned large language model for risk stratification and treatment guidance in hepatocellular carcinoma, aiming to improve precision therapy by leveraging electronic medical records.
MentalHospital is a virtual environment designed to evaluate AI agents and human experts on psychiatric clinical encounters, covering interviewing, examination, diagnosis, and treatment planning. Experiments compare human experts, trainees, and various LLMs on objective and subjective metrics.
EpistemeAI released Reasoning-Medical0.1-27B, a fine-tuned version of Qwen3.5-27B for medical reasoning, claiming to surpass MedGemma on several medical benchmarks by incorporating chain-of-thought reasoning on a curated dataset of 100,000 records.