AtomiMed: Hierarchical Atomic Fact-Checking for Universal Clinical-Aware Medical Report Evaluation
Summary
AtomiMed introduces a hierarchical atomic fact-checking framework for evaluating medical report generation, decomposing clinical narratives into atomic facts and using agentic cross-verification to improve accuracy assessment beyond traditional metrics.
View Cached Full Text
Cached at: 07/02/26, 03:46 AM
Paper page - AtomiMed: Hierarchical Atomic Fact-Checking for Universal Clinical-Aware Medical Report Evaluation
Source: https://huggingface.co/papers/2606.31292 Published on Jun 30
·
Submitted byhttps://huggingface.co/Venn2024
WANGon Jul 2
Authors:
,
,
,
,
,
,
,
,
,
Abstract
AtomiMed presents a novel evaluation framework for medical report generation that decomposes clinical narratives into atomic facts and uses an agentic cross-verification process to improve accuracy assessment beyond traditional metrics.
Traditional metrics forMedical Report Generation(MRG) predominantly rely on surface-level n-gram overlap, which fails to capture clinical factual accuracy and often overlooks catastrophic diagnostic errors. We address this fundamental limitation by proposing AtomiMed, a universal, modality-agnostic evaluation framework that decomposes complex medical narratives into a standardized, multi-level hierarchy ofAtomic Clinical Facts, encompassing Disease-level entities and Attribute-level descriptors, including location, morphology, and severity. By implementing anAgentic Cross-Verificationloop between ground-truth and predicted reports, AtomiMed simulates a multi-radiologist peer-review process to verify clinical consistency, thus enabling the decoupled assessment of diagnostic detection and descriptive accuracy. To facilitate standardized evaluation, we introduce MRGEvalKit, an open-source toolkit for automatedhierarchical extraction, and curate OmniMRG-Bench, a comprehensivemulti-modal benchmarkcovering X-ray, CT, MRI, and Ultrasound. Extensive experiments on multiple expert-annotated reader studies demonstrate that AtomiMed achieves significantly higher correlation with humanradiologist judgmentcompared to traditional and model-based metrics. Our code are release at https://github.com/Venn2336/MRGEvalkit
View arXiv pageView PDFGitHub2Add to collection
Get this paper in your agent:
hf papers read 2606\.31292
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.31292 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.31292 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.31292 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Generating Biomedical Fact-Checking Reports with RL-Enhanced Agentic Search
This paper introduces BioCheck Agent, an LLM-based agent that generates structured biomedical fact-checking reports using RL-enhanced agentic search, showing improved accuracy and reduced hallucinations compared to base models.
MedSNIP: Building and Benchmarking Snippet-Level Granularity for Medical Fact Verification
The paper introduces MedSNIP, a snippet-level approach for medical fact verification, and MedSNIP-Bench, a human-annotated benchmark, demonstrating improvements in preserving clinical structure and reducing verifier calls.
Multi-Agent AI System for Radiology Report Structuring and Quality Assurance with Independent Radiologist Evaluation
The paper presents a locally deployed multi-agent AI system for structuring radiology reports and performing quality assurance, with radiologist evaluation showing favorable performance.
AutoMedBench: Towards Medical AutoResearch with Agentic AI Models
AutoMedBench is a workflow-aware benchmark for autonomous medical-AI research, evaluating agents across five stages on diverse medical imaging tasks. Stage-level scoring reveals validation as the weakest stage, highlighting the need for reliable verification in agentic workflows.
A Modular Agent for Reliable and Auditable Spatial Relation Verification in CT Scans
This paper presents a modular medical imaging agent that verifies spatial relations in CT scans by decomposing tasks into parsing, localization, and geometric rules, achieving 94.1% accuracy and outperforming end-to-end vision-language models by 42.5 percentage points on a benchmark while ensuring auditable reasoning.