RadFusion: Towards Threshold-Controllable Radiology Report Generation

# RadFusion: Towards Threshold-Controllable Radiology Report Generation
Source: [https://arxiv.org/html/2608.10505](https://arxiv.org/html/2608.10505)
Ying Jin Noel C\. F\. Codella John Corring Mu Wei Dinei Florencio Eric Horvitz

Microsoft

\{ying\.jin, ncodella, john\.corring\}@microsoft\.com \{muhsin\.wei, dinei, horvitz\}@microsoft\.com

###### Abstract

Automated radiology report generation is advancing rapidly in response to the shortage of radiologists, yet unlike a perception model, existing generation models offer no control over the sensitivity–specificity trade\-off of their diagnostic content\. Such control is essential because clinical scenarios diverge: emergency triage prioritizes sensitivity to reduce missed findings, whereas confirmatory interpretation emphasizes specificity to limit unnecessary interventions\. A single fixed report can neither adapt to these scenarios nor support the ROC\-based validation widely expected for regulatory clearance\. We introduce RadFusion, a framework that equips report generation with threshold controllability\. Our method fuses a multi\-label classifier, which provides per\-disease confidence scores, with a VQA\-based report generator, which describes medical findings in detail; an LLM then rewrites the report so that its stated diagnoses follow the classifier’s decisions at the selected threshold while staying grounded in the generator’s descriptions\. On MIMIC\-CXR, the performance of RadFusion conforms to the classifier’s ROC curve: sweeping the threshold and mapping the reports back to class labels reproduces the classifier’s validated ROC performance\. This conformance makes generated reports quantitatively evaluable through ROC analysis, strengthening the case for regulatory clearance, and enables operating\-point selection that matches report behavior to clinical context\. Moreover, combining the two model types improves diagnostic accuracy over uncontrolled generation: sensitivity increases by 6\.9% at matched specificity, and specificity by 20\.7% at matched sensitivity\. These results show that RadFusion makes report generation clinically adaptable, quantitatively verifiable, and diagnostically more reliable\.

## 1Introduction

Interpreting medical images and communicating the findings in written reports is a central task in clinical care, and the growing shortage of radiologists makes its automation increasingly valuable\. Clinical use, however, demands more than a fixed, plausible report: diagnostic behavior must adapt to diverse clinical scenarios and support regulatory validation\. Perception models such as image classifiers meet these demands because their numerical confidence scores can be thresholded to trade off sensitivity against specificity and support ROC\-based validation widely used in regulatory approval processes such as FDA clearance\(Food,[2007](https://arxiv.org/html/2608.10505#bib.bib218)\)\. This threshold control is equivalent to choosing an operating point on the model’s ROC curve, serving diverse clinical needs: emergency triage calls for low thresholds that prioritize sensitivity to reduce missed findings, whereas confirmatory assessments call for high thresholds that prioritize specificity to limit unnecessary interventions\. However, perception models are constrained by predefined classes: a classifier may assign an 87% probability to pneumonia, but cannot describe its anatomical location, extent, or underlying cause\. Report generation models offer complementary capabilities by producing expressive, open\-vocabulary text that is standard in clinical reporting practice\. However, these models lack confidence estimates or threshold\-based mechanisms, making them difficult to accurately assess and adjust, and ultimately leaving a gap toward trustworthy, deployable clinical AI\(Linet al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib379)\)\.

Radiology report generation is typically framed as an image captioning task and is advancing rapidly with recent progress in large vision–language models\(Chaveset al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib242); Hylandet al\.,[2023](https://arxiv.org/html/2608.10505#bib.bib244); Chenet al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib212); Zhouet al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib189); Zhanget al\.,[2025](https://arxiv.org/html/2608.10505#bib.bib190); Tuet al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib245)\)\. Despite the improved performance, this captioning formulation yields a single, fixed report with no mechanism to adjust the diagnostic operating point\. Although controllability and confidence estimation for text generation have been studied in the general domain, existing work does not control the generated text through class\-specific thresholds\. For example, one line of work\(Dathathriet al\.,[2019](https://arxiv.org/html/2608.10505#bib.bib356); Yang and Klein,[2021](https://arxiv.org/html/2608.10505#bib.bib369)\)studies controllable text generation, which steers generation toward desired attributes at the token level, and another line of work\(Kadavathet al\.,[2022](https://arxiv.org/html/2608.10505#bib.bib229); Xionget al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib370)\)focuses on the confidence\-based self\-assessment of candidate answers\. To the best of our knowledge, threshold\-controllable text generation remains a critical yet under\-explored problem for clinical deployment and regulatory approval\.

To fill this gap, we introduce RadFusion, a threshold\-controllable radiology report generation framework that aligns the diagnostic content of generated reports with the ROC characteristics of a medical image classifier\. RadFusion fuses three components: \(1\) a classification model that produces calibrated per\-class confidence scores, \(2\) a VQA\-based report generation model that produces detailed radiology reports and supports follow\-up queries for additional details, and \(3\) an off\-the\-shelf large language model that rewrites the report so that its stated diagnoses are consistent with threshold\-adjusted classification, while remaining grounded in the report generation model’s findings to preserve clinical specificity\. By varying the thresholds, the system produces reports spanning a range of diagnostic operating points, each corresponding to a distinct sensitivity–specificity trade\-off\. Moreover, since the output text is factually consistent with the classifier, the numerical confidence estimates serve as a complement to the textual report, enabling ROC\-based regulatory validation and easing integration into automated systems\.

RadFusion is a portable framework, generally applicable to a wide range of classifiers and report generation models\. In this paper, we further study how various implementations of each component affect the final performance and identify the optimal configuration through systematic comparison\. As a result, experiments on MIMIC\-CXR\(Johnsonet al\.,[2019](https://arxiv.org/html/2608.10505#bib.bib128)\)show that our threshold\-adjusted reports exhibit the expected concave ROC behavior, with area under the curve closely tracking that of the underlying classifier\. In addition, threshold control yields improvements in diagnostic accuracy relative to uncontrolled generation, including a 6\.9% increase in sensitivity at matched specificity and a 20\.7% increase in specificity at matched sensitivity\. Collectively, these results demonstrate three contributions of this work:

- •Clinical adaptability: clinician\-customizable reporting allows operating\-point selection to match the clinical context\.
- •Regulatory validation: ROC\-based evaluation of generated reports provides a viable path toward regulatory approval\.
- •Diagnostic accuracy: fusing complementary model types improves diagnostic accuracy over uncontrolled generation\.

## 2Related Work

Radiology report generation\.Early work adapted encoder–decoder architectures to radiology report generation: R2Gen\(Chenet al\.,[2020](https://arxiv.org/html/2608.10505#bib.bib209)\)introduced memory\-driven transformers, R2GenCMN\(Chenet al\.,[2022](https://arxiv.org/html/2608.10505#bib.bib107)\)added cross\-modal memory, and CvT2DistilGPT2\(Nicolsonet al\.,[2023](https://arxiv.org/html/2608.10505#bib.bib211)\)combined vision transformers with distilled decoders\. Recent approaches leverage large vision–language models, including LLaVA\-Rad\(Chaveset al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib242)\), MAIRA\(Hylandet al\.,[2023](https://arxiv.org/html/2608.10505#bib.bib244); Bannuret al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib363)\), CheXagent\(Chenet al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib212)\), MedVersa\(Zhouet al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib189)\), Libra\(Zhanget al\.,[2025](https://arxiv.org/html/2608.10505#bib.bib190)\), MedGemma\(Sellergrenet al\.,[2025](https://arxiv.org/html/2608.10505#bib.bib258)\), and Med\-PaLM M\(Tuet al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib245)\)\. Complementary strategies, including knowledge\-driven modeling\(Liet al\.,[2019](https://arxiv.org/html/2608.10505#bib.bib136); Zhanget al\.,[2020](https://arxiv.org/html/2608.10505#bib.bib377)\), prompt\-based methods\(Jinet al\.,[2024a](https://arxiv.org/html/2608.10505#bib.bib227)\), region\-guided generation\(Tanidaet al\.,[2023](https://arxiv.org/html/2608.10505#bib.bib257)\), and retrieval augmentation\(Sunet al\.,[2025](https://arxiv.org/html/2608.10505#bib.bib378)\), have improved factual grounding\.QRad\([Jinet al\.,](https://arxiv.org/html/2608.10505#bib.bib362)\), a state\-of\-the\-art model based on a pre\-trainedMedImageInsight\(Codellaet al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib246)\)encoder, unifies the report generation and medical VQA tasks, and serves as the report generation component of our framework\. All these models, however, produce a single, fixed report with no mechanism to adjust diagnostic behavior or select an operating point\.

Medical visual question answering\.Medical VQA offers an interface for extracting clinical information via natural language queries\(Lauet al\.,[2018](https://arxiv.org/html/2608.10505#bib.bib380); Heet al\.,[2020](https://arxiv.org/html/2608.10505#bib.bib381); Liuet al\.,[2021](https://arxiv.org/html/2608.10505#bib.bib382)\)\. Multimodal models such as Med\-Flamingo\(Mooret al\.,[2023](https://arxiv.org/html/2608.10505#bib.bib291)\)and LLaVA\-Med\(Liet al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib241)\)adapt vision–language architectures to medicine, while Rad\-ReStruct\(Pellegriniet al\.,[2023a](https://arxiv.org/html/2608.10505#bib.bib278)\)proposed hierarchical VQA for structured reporting and RaDialog\(Pellegriniet al\.,[2023b](https://arxiv.org/html/2608.10505#bib.bib279)\)combined report generation with multi\-turn dialogue\. Since omissions are common in radiology reports, our framework uses medical VQA to recover omitted information from the image, supplying grounded descriptions when a low threshold flips a finding to positive\.

Controllable text generation\.Controllable generation methods such as PPLM\(Dathathriet al\.,[2019](https://arxiv.org/html/2608.10505#bib.bib356)\), FUDGE\(Yang and Klein,[2021](https://arxiv.org/html/2608.10505#bib.bib369)\), and Classifier\-free Guidance\(Ho and Salimans,[2022](https://arxiv.org/html/2608.10505#bib.bib374)\)steer token probabilities during decoding toward desired attributes\. While effective for stylistic or semantic control, these methods do not support class\-specific diagnostic thresholds or operating\-point selection\.

Confidence estimation and calibration\.Existing work on confidence estimation for LLMs includes logit\-based methods usingP​\(True\)P\(\\text\{True\}\)\(Kadavathet al\.,[2022](https://arxiv.org/html/2608.10505#bib.bib229)\), verbalized confidence\(Xionget al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib370); Liet al\.,[2025](https://arxiv.org/html/2608.10505#bib.bib277)\), sampling\-based self\-consistency\(Wanget al\.,[2023](https://arxiv.org/html/2608.10505#bib.bib371)\), and the information geometry of chain\-of\-thought outputs\(Lauet al\.,[2025](https://arxiv.org/html/2608.10505#bib.bib6)\)\. Calibration then aligns the predicted confidence with empirical correctness, typically through post\-hoc temperature scaling\(Guoet al\.,[2017](https://arxiv.org/html/2608.10505#bib.bib372); Platt,[1999](https://arxiv.org/html/2608.10505#bib.bib373)\)or proper scoring rules during training\(Blasioket al\.,[2023](https://arxiv.org/html/2608.10505#bib.bib275); Fröhlich and Williamson,[2024](https://arxiv.org/html/2608.10505#bib.bib276)\); recent task\-specific methods partition the input space with uniform mass binning to approximate Bayes\-optimal calibrators\(Manggalaet al\.,[2025](https://arxiv.org/html/2608.10505#bib.bib5)\)\. For radiology report generation,Wanget al\.\([2024](https://arxiv.org/html/2608.10505#bib.bib375)\)considered confidence during training by weighting losses with uncertainty, but offered no control mechanism at inference\. Our framework builds on these foundations: per\-class confidence scores are calibrated via temperature scaling so that thresholds map predictably to operating points, and logit\-basedP​\(Yes\)P\(\\text\{Yes\}\)extraction offers an alternative classifier implementation\.

## 3Method

### 3\.1Overview

RadFusion enables threshold\-controllable report generation by*conforming*the diagnostic decisions of the generated report to those of a classification model operating at a chosen threshold, while preserving the generative model’s clinical expressiveness\. It is a portable framework that is applicable to a wide range of classifiers and report generation models\. In this section, we first introduce the implementation that yields the optimal overall performance, followed by alternative designs for each component; the comparison among these implementations is discussed in[Section˜5](https://arxiv.org/html/2608.10505#S5)\.

![Refer to caption](https://arxiv.org/html/2608.10505v1/figs/tc-fig1.png)Figure 1:Overview of RadFusion, our threshold\-controllable report generation framework\. The classification model produces confidence scores for predefined disease categories, which are thresholded to binary decisions\. The report generation model produces an initial report\. The LLM rewriter combines both outputs so that the report’s stated diagnoses conform to the binary classifications while preserving clinical detail\. We useQRad, a report generation model supporting follow\-up questions to obtain evidence omitted from the initial report\.As illustrated in[Figure˜1](https://arxiv.org/html/2608.10505#S3.F1), RadFusion fuses three components:

1. 1\.Perception model\([Section˜3\.2](https://arxiv.org/html/2608.10505#S3.SS2)\): Given a chest X\-ray image, we use a classifier to produce numerical confidence scores\{P​\(Ci=1\)\}i=1K\\\{P\(C\_\{i\}=1\)\\\}\_\{i=1\}^\{K\}forKKpredefined disease classes\. Given a thresholdτ\\tau, it yields binary predictionsy^i=𝟏​\[P​\(Ci=1\)\>τ\]\\hat\{y\}\_\{i\}=\\mathbf\{1\}\[P\(C\_\{i\}=1\)\>\\tau\], partitioning classes into positive \(disease present\) and negative \(disease absent\)\.
2. 2\.Report generation model\([Section˜3\.3](https://arxiv.org/html/2608.10505#S3.SS3)\): Given the same image, a report generation model produces a free\-text radiology reportrrdescribing both positive and negative findings\. Compared to the classifier’s binary labels, this open\-vocabulary report contains richer information, including the anatomical location, severity, progression, and clinical context of each finding\.
3. 3\.LLM rewriter\([Section˜3\.4](https://arxiv.org/html/2608.10505#S3.SS4)\): An off\-the\-shelf large language model that combines the classifier’s binary decisionsy^\\hat\{y\}with the report’s detailed descriptionsrr, rewriting the report so that its stated diagnoses align with the thresholded classes while remaining grounded in the clinical attributes from the original report\.

By sweeping a thresholdτ\\taufrom 0 to 1 per class, RadFusion produces a family of reports spanning the full sensitivity–specificity spectrum\. For evaluation, we assess the diagnostic quality of these reports across operating points with a closed\-loop protocol detailed in[Section˜4\.2](https://arxiv.org/html/2608.10505#S4.SS2)\.

### 3\.2Perception Model: MedImageInsight Classifier

Given a chest X\-ray image, the classification model produces numerical confidence scores for 14 disease classes defined by CheXpert\(Irvinet al\.,[2019](https://arxiv.org/html/2608.10505#bib.bib225)\), such as Enlarged Cardiomediastinum, Cardiomegaly, and Lung Opacity\. We studythree differentclassification approaches\. The baseline approach is based onMedImageInsight\(MI2\)\(Codellaet al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib246)\), a medical image\-text foundation model with a DaViT\(Dinget al\.,[2022](https://arxiv.org/html/2608.10505#bib.bib299)\)image encoder \(360M parameters\) and a language encoder \(252M parameters\), pre\-trained via contrastive learning\(Yanget al\.,[2022](https://arxiv.org/html/2608.10505#bib.bib301)\)on over 3\.5 million medical image\-text pairs across 12 imaging domains\. A classification model is fine\-tuned and calibrated on MIMIC\-CXR\(Johnsonet al\.,[2019](https://arxiv.org/html/2608.10505#bib.bib128)\)as described below\. Its performance evaluation is discussed in[Section˜4](https://arxiv.org/html/2608.10505#S4)\.

Classifier Approach 1: Baseline\.To adapt MI2 to the CheXpert classes, we fine\-tune it with the Image\-Text\-Class Hybrid Contrastive loss\(Jinet al\.,[2024b](https://arxiv.org/html/2608.10505#bib.bib360)\)\. This hybrid loss enables the model to jointly leverage image\-class and image\-text supervision to attain superior classification performance\. The structured class labels ground the model on the disease classes of interest, while the free\-text reports provide richer, fine\-grained supervision that prevents the feature space from collapsing onto the coarse class labels alone\. The loss function is defined as:

ℒ=ℒimage\-text\+λ​ℒimage\-class,\\mathcal\{L\}=\\mathcal\{L\}\_\{\\text\{image\-text\}\}\+\\lambda\\mathcal\{L\}\_\{\\text\{image\-class\}\},\(1\)whereℒimage\-text\\mathcal\{L\}\_\{\\text\{image\-text\}\}is the image\-text contrastive loss based on softmax cross\-entropy, andℒimage\-class\\mathcal\{L\}\_\{\\text\{image\-class\}\}is the image\-class contrastive loss based on sigmoid cross\-entropy \(the sigmoid accommodates the multi\-label setup\)\. The embeddings for the classes are based on template prompts \(e\.g\., “a chest X\-ray showing \[CLASS NAME\]”\)\. We use class labels from VisualCheXbert\(Jainet al\.,[2021](https://arxiv.org/html/2608.10505#bib.bib124)\)\. At inference, the class probabilities are computed as:

P​\(Ci=1∣I\)=sigmoid​\[\(cosine\_similarity​\(fI​\(I\),fT​\(ti\)\)/τtemp\)\],P\(C\_\{i\}=1\\mid I\)=\\text\{sigmoid\}\[\(\\text\{cosine\\\_similarity\}\(f\_\{I\}\(I\),\\;f\_\{T\}\(t\_\{i\}\)\)/\\tau\_\{\\text\{temp\}\}\)\],\(2\)wherefIf\_\{I\},fTf\_\{T\}are the image and text encoders,tit\_\{i\}is the class prompt, andτtemp\\tau\_\{\\text\{temp\}\}is a learnable temperature\. Since the class prompts are fixed, we precomputefT​\(ti\)f\_\{T\}\(t\_\{i\}\)as the weights of a single linear classification head, removing the text encoder at inference time\.

The confidence scores are calibrated by temperature scaling\(Guoet al\.,[2017](https://arxiv.org/html/2608.10505#bib.bib372)\), which involves learning a scalarTTthat minimizes the Expected Calibration Error on a validation set\. This lightweight post\-hoc correction improves calibration without altering discriminative performance \(AUC is unchanged\)\.

Classifier Approach 2: Linear probing of QRad’s encoder\.An alternative implementation of the classifier trains a linear classification head on top ofQRad’s frozen image encoder, i\.e\., a linear probe\. SinceQRad’s encoder is also initialized from MI2, this alternative differs from the MI2 classifier only by an additional generative fine\-tuning step; moreover, at test time the classifier and the report generation model share the same encoder, reducing computational cost\. The linear probe boosts performance on hard\-to\-detect classes \(*e\.g*\.Pneumothorax\) but slightly underperforms on easy\-to\-detect ones \(*e\.g*\.Support Devices\), with overall ROC\-AUC comparable to the MI2 classifier, as shown in[Section˜5](https://arxiv.org/html/2608.10505#S5)\.

Classifier Approach 3: Single\-token probability from QRad\.QRad, being a hybrid report generation and medical VQA model, is trained with binary classification questions answered by single\-token yes/no responses \(e\.g\., “Is this image classified as \[CLASS\]? \(yes/no\)”\)\. We extract the softmax probability over the\[Yes\]and\[No\]logits\(Kadavathet al\.,[2022](https://arxiv.org/html/2608.10505#bib.bib229)\)as the class confidence\. This alternative has the advantage of using a single model for both numerical confidence and textual report generation, but it underperforms a standalone classifier on most classes\.

### 3\.3Report Generation Model: QRad Auto\-VQA

Given a chest X\-ray image, a report generation model produces a free\-text radiology report that describes the patient’s findings in clinical language\. The report covers both positive findings \(e\.g\., “mild bibasilar atelectasis”\) and pertinent negatives \(e\.g\., “no pneumothorax”\), along with attributes that binary labels cannot express, such as the*location*of a finding \(right lower lobe vs\. bilateral\), its*severity*\(mild, moderate, severe\),*progression*relative to prior studies \(stable, worsening, improved\), and*clinical context*\(e\.g\., post\-surgical changes\)\. Omission of some negative findings is common in radiology reports\.

Conventional report generation models follow a direct image\-to\-text mappingY=f​\(I\)Y=f\(I\), producing a monolithic report with no mechanism to query additional details that may have been omitted\. We useQRad\([Jinet al\.,](https://arxiv.org/html/2608.10505#bib.bib362)\), which builds on the MI2 vision encoder and reframes report generation as a self\-directed Visual Question Answering \(Auto\-VQA\) process:

Q=fQ​\(I\),Y=fA​\(I,Q\),Q=f\_\{Q\}\(I\),\\quad Y=f\_\{A\}\(I,Q\),\(3\)where a Question GeneratorfQf\_\{Q\}predicts a sequence of clinically relevant questions conditioned on the image, and an Answer GeneratorfAf\_\{A\}answers each question to produce one sentence of the report\. The final report is the concatenation of all answers\. This Auto\-VQA reframing allows RadFusion to query disease categories omitted in the initial report by composing questions and sending them to the Answer GeneratorfAf\_\{A\}: when a low threshold flips a class to positive, a follow\-up query returns a grounded description of the finding with its location, severity, and context, rather than leaving the rewriter to hallucinate such details\. The initial report and all follow\-up answers together form the evidence pool for rewriting\.

Alternative approaches:RadFusion is widely applicable to report generation models, such as UniRG\-CXR\(Liuet al\.,[2026](https://arxiv.org/html/2608.10505#bib.bib365)\), which leverages reinforcement fine\-tuning\. For models without a query interface, a separate VQA model may be required to recover omitted information\. BecauseQRad serves both roles with a single model, we adoptQRad in this study as a simple yet effective implementation\.

### 3\.4LLM Rewriting

The LLM rewriter combines the outputs of the two preceding components: the binary predictions\{y^i\}i=1K\\\{\\hat\{y\}\_\{i\}\\\}\_\{i=1\}^\{K\}from the thresholded classifier, and the evidence poolℰ\\mathcal\{E\}consisting of the initial reportrrand optional follow\-up answers\. We use an off\-the\-shelf LLM such as GPT\-5\(Singhet al\.,[2025](https://arxiv.org/html/2608.10505#bib.bib3)\)to rewrite the report111We use an on\-prem deployment to satisfy dataset usage agreements\.because modern LLMs have demonstrated strong instruction\-following capabilities, avoiding the need to train a dedicated model for this task\.

The rewriting instruction for the LLM plays a critical role in ensuring that its stated diagnoses are consistent withy^i\\hat\{y\}\_\{i\}while remaining grounded in the clinical attributes \(location, severity, progression,*etc*\.\) provided byℰ\\mathcal\{E\}\. Therefore, we structure the instruction to explicitly define the desired behavior of the LLM, including how to handle positive and negative classes, how to incorporate evidence fromℰ\\mathcal\{E\}, and how to maintain non\-class content\. Specifically:

- •For each positive class \(y^i=1\\hat\{y\}\_\{i\}=1\): if the finding is already present in the report, it is kept unchanged; if absent or contradicted, the LLM adds or corrects it using detail fromℰ\\mathcal\{E\}\.
- •For each negative class \(y^i=0\\hat\{y\}\_\{i\}=0\): if the finding is mentioned as present, the LLM removes or negates it; if already absent, no change is needed\.
- •Non\-class content \(e\.g\., imaging quality, support devices not in the class set\) is preserved\.

An LLM capable of this task must have medical domain knowledge to reason over the disease class names and the input report\. To help establish the correspondence between classes and their varied natural language manifestations in the report, we provide examples per class as part of the instruction\. For example, we show that the class “Pleural Effusion” may appear in a report as “small bilateral pleural effusions”, “fluid in the costophrenic angles”, or “blunting of the costophrenic recesses”\. The full instruction is provided in[Appendix˜E](https://arxiv.org/html/2608.10505#A5)\.

Alternative approaches: other LLMs as the rewriter\.RadFusion can employ any capable LLM as the rewriter, and satisfactory performance depends on the model’s medical knowledge, reasoning, and instruction\-following capabilities\. In[Section˜5](https://arxiv.org/html/2608.10505#S5), we compare alternatives spanning models of varied sizes, including GPT 5\.4, GPT 5\.4\-mini, and DeepSeek V4 Pro, as well as varied levels of reasoning effort\.

### 3\.5Extension to Three\-Dimensional Threshold Control for Clinical Deployment

RadFusion’s rewriting\-based design extends naturally beyond a single binary threshold\. In clinical practice, radiologists rarely operate on a strict positive/negative dichotomy: borderline findings are reported with explicit hedging \(e\.g\., “cannot exclude early consolidation”\), and the appropriate response to a finding depends not only on diagnostic confidence but also on its time sensitivity\. To support such graded decision\-making, we extend the threshold space to three dimensions\(Ta,Tb,Tc\)\(T\_\{a\},T\_\{b\},T\_\{c\}\)per class, which requires only modified rewriting instructions rather than any model retraining\. The first two dimensions \(Ta≤TbT\_\{a\}\\leq T\_\{b\}\) partition the confidence spectrum into three zones:

- •P​\(Ci=1∣I\)<TaP\(C\_\{i\}=1\\mid I\)<T\_\{a\}:*negative zone*, where the report denies the finding\.
- •P​\(Ci=1∣I\)\>TbP\(C\_\{i\}=1\\mid I\)\>T\_\{b\}:*positive zone*, where the report asserts the finding\.
- •Ta≤P​\(Ci=1∣I\)≤TbT\_\{a\}\\leq P\(C\_\{i\}=1\\mid I\)\\leq T\_\{b\}:*semi\-positive zone*, where the report describes the finding while explicitly flagging uncertainty and recommending further evaluation\.

The third dimension,TcT\_\{c\}, thresholds a per\-class*urgency*score, motivated by the fact that clinical urgency does not always align with imaging severity \(e\.g\., an early pneumothorax may demand immediate treatment\); the score can be derived from clinical priors, patient context, or an auxiliary classifier, and findings exceedingTcT\_\{c\}are emphasized as time\-sensitive and actionable, independent of theTaT\_\{a\}/TbT\_\{b\}axis\. The system presented in[Sections˜3\.1](https://arxiv.org/html/2608.10505#S3.SS1)and[3\.4](https://arxiv.org/html/2608.10505#S3.SS4)is the special case withTa=TbT\_\{a\}=T\_\{b\}andTcT\_\{c\}omitted, which we adopt for direct ROC\-based comparison against the underlying classifier\. This extension illustrates the extendability of the design: richer reporting policies, such as asserting findings conservatively \(highTbT\_\{b\}\) while escalating asserted ones aggressively \(lowTcT\_\{c\}\), are implemented by adjusting thresholds and instructions alone\. We view clinician\-tunable operating\-point interfaces as a promising direction for future work\.

## 4Experiments

### 4\.1Dataset and Setup

We conduct experiments on MIMIC\-CXR\(Johnsonet al\.,[2019](https://arxiv.org/html/2608.10505#bib.bib128)\), one of the largest radiology report datasets containing 227,835 studies with 377,110 chest X\-ray images\. Following standard practice\(Chaveset al\.,[2024](https://arxiv.org/html/2608.10505#bib.bib242); Hylandet al\.,[2023](https://arxiv.org/html/2608.10505#bib.bib244)\), we use the Findings section as the generation target\. All evaluations use the official test split \(2,347 studies\)\.

### 4\.2Evaluation Methods

To assess the threshold controllability of RadFusion, we implement a closed\-loop evaluation protocol that runs from the classifier’s thresholded decisions, through the rewritten report, and back to class labels, which are then compared against the original decisions for alignment\. Given a thresholdτ\\tau, the classifier’s confidence scores are binarized into positive and negative class lists, and the LLM rewritesQRad’s initial report to conform to these classifications, as described in[Section˜3\.4](https://arxiv.org/html/2608.10505#S3.SS4)\. To evaluate, we convert the rewritten report back to binary class labels using GPT\-5, which we found more reliable than VisualCheXbert\(Jainet al\.,[2021](https://arxiv.org/html/2608.10505#bib.bib124)\), a BERT\-based text classification model commonly used to extract disease classes from radiology reports\. Comparing the extracted labels against the ground truth yields the true positive rate \(sensitivity\) and false positive rate \(1−specificity1\-\\text\{specificity\}\) of the rewritten reports at thresholdτ\\tau\.

We sweepτ\\taufrom 0\.0 to 1\.0 with a step size of 0\.1, yielding 11 operating points\.222We find 0\.1 is a reasonable step size for the threshold\. Smaller step sizes produce smoother curves, but increase LLM inference cost linearly\.The sameτ\\tauis applied across all 13 disease classes \(“No Finding” is excluded because it semantically overlaps with the absence of other classes\)\. Connecting these operating points traces the ROC curve of the threshold\-controlled reports\. The ROC curve plots TPR against FPR across thresholds, and each of its points corresponds to a specific sensitivity–specificity trade\-off, making ROC the natural evaluation for threshold\-controllable systems\. It is also the metric widely expected by regulatory approval processes for medical devices\. The area under the curve \(AUC\) summarizes the discriminative performance across all operating points on the ROC curve\. We compare both the ROC curves and their AUC among three systems \([Figures˜2](https://arxiv.org/html/2608.10505#S4.F2)and[4](https://arxiv.org/html/2608.10505#A2.F4)\):

- •The classifier \(blue curves\): ROC computed directly from the classifier’s continuous confidence scores, the upper bound we aim to conform to\.
- •Threshold\-controlled reports \(red curves\): ROC computed by converting each threshold\-adjusted report back to class labels\. Each point corresponds to one operating point specified by a threshold\.
- •Non\-controlled reports \(green cross\): The single operating point for the generated report without threshold adjustments\.

![Refer to caption](https://arxiv.org/html/2608.10505v1/x1.png)Figure 2:Threshold\-controllable report generation on four representative classes\.Cardiomegaly: common, both models perform strongly\.Edema: threshold tuning is clinically valuable since mild cases often need no treatment\.Atelectasis: challenging, requires fine visual details\.Pleural Other: rare \(1\.3% prevalence\)\. Across all four, the report ROC \(red\) closely tracks the classifier \(blue\), and the non\-controlled report \(green cross\) falls below the curves\.
### 4\.3Results and Discussion

Overall, the results demonstrate that RadFusion delivers effective threshold control: the threshold\-controlled reports closely conform to the classifier’s ROC curves, improve diagnostic accuracy over non\-controlled generation, and expose a tunable sensitivity–specificity trade\-off that serves diverse clinical scenarios\. We highlight three key findings:

Conformance to classifier ROC\.[Figure˜2](https://arxiv.org/html/2608.10505#S4.F2)highlights four classes spanning a range of difficulty and prevalence: Cardiomegaly \(common\), Edema \(threshold\-sensitive\), Atelectasis \(hard\), and Pleural Other \(rare, 1\.3%\)\. Across all four, the report ROC closely tracks the classifier ROC with comparable AUC, and the concave shape confirms a monotonic sensitivity–specificity trade\-off; the full 13\-class evaluation \([Figure˜4](https://arxiv.org/html/2608.10505#A2.F4)\) shows the same pattern\. This close overlap validates that the threshold effectively trades off sensitivity and specificity, and that the rewritten reports conform to the classifier’s decisions across the operating range\.

Improved factual correctness\.The ROC curves of threshold\-controlled reports are*above*the non\-controlled reports \(green crosses\): at matched specificity, sensitivity improves by 6\.9%; at matched sensitivity, specificity improves by 20\.7%\. By combining the two model types, threshold control transfers the classifier’s superior discriminative performance to the generated reports\.

Clinical utility\.Low thresholds prioritize sensitivity \(screening/triage\); high thresholds prioritize specificity \(confirmatory assessment\)\. Our framework makes this trade\-off explicit and tunable, replacing the fixed, opaque operating point of conventional models and offering a path toward ROC\-based regulatory evaluation of generated reports\.

## 5Ablation Studies

### 5\.1Alternative Classifier Implementations

Here we compare the three classifier implementations detailed in[Section˜3\.2](https://arxiv.org/html/2608.10505#S3.SS2): a standalone classifier fine\-tuned from MI2 \(our default\), a linear probe ofQRad’s frozen encoder \(sharing the encoder with report generation\), and single\-token yes/no probabilities extracted fromQRad’s VQA answers \(a single model for classification and report generation\)\. All three implementations use the sameMedImageInsightpre\-trained image encoder, andQRad further fine\-tunes the encoder for report generation\.

[Table˜1](https://arxiv.org/html/2608.10505#S5.T1)reports their per\-class AUC\-ROC, and their ROC curves are provided in[Appendix˜C](https://arxiv.org/html/2608.10505#A3)\. The MI2 classifier and theQRad linear probe achieve comparable overall AUC, each stronger on a different subset of classes, while theQRad token\-logit variant trails on most classes\. Notably, theQRad Linear Prob implementation significantly improves on hard classes like Pneumothorax, possibly because the report generation objective enhances the encoder’s visual representation to better capture subtle visual cues\. The MI2 FT classifier is stronger on visually obvious classes like Support Devices, possibly due to its pretraining on a large corpus of radiology images\. RadFusion uses the MI2 FT classifier by default to demonstrate a common use case where separate classifiers and report generators are combined for better utility than either alone\.

Table 1:AUC\-ROC of the three classifier implementations\. MI2 FT: the fine\-tuned classifier based onMedImageInsight;QRad Linear Prob: fitting a linear classification head onQRad’s frozen encoder;QRad Token Logit: single\-token yes/no probabilities usingQRad’s VQA capability\. Results show AUC\-ROC across the 13 disease classes, with AVG their macro average\.AUC\-ROCEnl\.Car\.L\.O\.L\.L\.Ede\.Con\.Pmn\.Ate\.Pmt\.P\.E\.P\.O\.Fra\.S\.D\.AVGMI2 FT\.93\.92\.92\.85\.92\.91\.88\.90\.88\.90\.90\.89\.95\.90QRad Linear Prob\.92\.91\.92\.89\.91\.91\.89\.91\.95\.92\.90\.87\.91\.91QRad Token Logit\.82\.81\.83\.73\.78\.78\.71\.74\.68\.78\.58\.56\.65\.73
### 5\.2Alternative LLM Rewriters

Model size and rewriting instruction\.We compare LLM rewriters across model sizes \(GPT\-5, GPT\-5\.4, and GPT\-5\.4\-mini\) and rewriting instructions, with and without the per\-class example text segments \([Section˜3\.4](https://arxiv.org/html/2608.10505#S3.SS4)\); by default, the instruction includes the examples\.[Table˜2](https://arxiv.org/html/2608.10505#S5.T2)reports the AUC\-ROC of the threshold\-controlled reports for each configuration\. Removing the examples degrades performance: for GPT\-5, the AUC\-ROC on Fracture \(Fra\.\) drops from \.89 to \.73 and the macro average from \.90 to \.88\. This confirms that grounding the class–text correspondence with concrete examples is important for faithful rewriting, consistent with our analysis in[Section˜3\.4](https://arxiv.org/html/2608.10505#S3.SS4)\. We further observe that a newer model generation does not necessarily outperform its predecessor under the same prompt: GPT\-5\.4 does not exceed GPT\-5\. This suggests that different models may require specifically optimized instructions for the optimal performance \(the instruction was tuned on GPT\-5\)\.

Table 2:Comparison of LLM rewriter implementations across model sizes and rewriting instructions\. By default, the instruction includes the per\-class example text segments; “w/o examples” removes them\. The table shows AUC\-ROC of the threshold\-controlled reports across the 13 disease classes, with AVG their macro average\. Within each model group \(separated by midrules\), the higher AVG is in bold\.ModelReasoningEnl\.Car\.L\.O\.L\.L\.Ede\.Con\.Pmn\.Ate\.Pmt\.P\.E\.P\.O\.Fra\.S\.D\.AVGGPT\-5default\.93\.92\.92\.84\.92\.91\.88\.90\.88\.90\.90\.89\.93\.90GPT\-5 w/o examplesdefault\.88\.92\.91\.84\.91\.91\.88\.90\.88\.90\.90\.73\.89\.88GPT\-5\.4medium\.93\.92\.92\.75\.92\.91\.88\.90\.88\.90\.90\.89\.94\.90GPT\-5\.4\-minimedium\.93\.92\.92\.82\.90\.90\.88\.90\.84\.90\.90\.88\.93\.89

Model provider and reasoning effort\.[Table˜3](https://arxiv.org/html/2608.10505#S5.T3)extends the comparison across model providers \(OpenAI and DeepSeek\) and reasoning efforts, using theQRad Linear Prob classifier as the perception model for additional context\. The results exhibit a consistent pattern with[Table˜2](https://arxiv.org/html/2608.10505#S5.T2), and further show that reasoning effort is vital, especially for smaller models\. Notably, both GPT\-5\.4 and GPT\-5\.4\-mini experience significant degradation on Lung Lesion \(L\.L\.\), but raising the reasoning effort to medium improves GPT\-5\.4\-mini from \.58 to \.81 on this class\.

Table 3:Comparison of LLM rewriter implementations across model providers \(OpenAI and DeepSeek\) and reasoning efforts, using theQRad Linear Prob classifier as the perception model for additional context\. The pattern is consistent with[Table˜2](https://arxiv.org/html/2608.10505#S5.T2), and reasoning effort is vital especially for smaller models\. The table shows AUC\-ROC of the threshold\-controlled reports across the 13 disease classes, with AVG their macro average\. Within each model group \(separated by midrules\), the higher AVG is in bold\. DS\-V4\-Pro denotes DeepSeek V4 Pro\(Xuet al\.,[2026](https://arxiv.org/html/2608.10505#bib.bib387)\)\.ModelReasoningEnl\.Car\.L\.O\.L\.L\.Ede\.Con\.Pmn\.Ate\.Pmt\.P\.E\.P\.O\.Fra\.S\.D\.AVGGPT\-5default\.92\.91\.91\.85\.91\.90\.88\.90\.94\.91\.89\.86\.89\.90GPT\-5\.4default\.92\.91\.91\.73\.91\.90\.88\.90\.94\.91\.87\.84\.85\.88GPT\-5\.4medium\.92\.91\.91\.73\.91\.90\.89\.90\.94\.91\.89\.86\.88\.89GPT\-5\.4\-minidefault\.86\.89\.83\.58\.77\.63\.67\.85\.65\.85\.85\.56\.85\.76GPT\-5\.4\-minimedium\.92\.91\.91\.81\.90\.90\.87\.90\.88\.90\.89\.85\.88\.89DS\-V4\-Prodefault\.91\.91\.89\.77\.89\.89\.85\.89\.87\.89\.87\.85\.85\.87

## 6Conclusion

We introduced RadFusion, a threshold\-controllable radiology report generation framework that fuses a classifier, a VQA\-based report generator, and an LLM rewriter\. By conforming the report’s diagnostic content to the classifier’s threshold\-adjusted predictions, RadFusion produces reports whose performance tracks the classifier’s ROC curve while preserving rich clinical detail, so that a single threshold selects the operating point along the sensitivity–specificity spectrum\. On MIMIC\-CXR, this conformance improves sensitivity by 6\.9% at matched specificity and specificity by 20\.7% at matched sensitivity over non\-controlled generation\. To our knowledge, this is the first framework to achieve threshold control in report generation, making generated reports clinically adaptable, quantitatively evaluable, and diagnostically more accurate, and offering a practical path toward regulatory validation and clinician\-customizable AI\-assisted radiology\.

## 7Broader Impact

RadFusion can improve the safety and clinical utility of AI\-assisted radiology\. By making the sensitivity–specificity trade\-off explicit and tunable, it lets clinicians match model behavior to a clinical context \(high sensitivity for screening, high specificity for confirmatory assessment\) rather than relying on a fixed, opaque operating point, enabling responsible deployment across settings with diverse risk tolerances\. Because each report is paired with the classifier’s per\-class confidence scores, the framework also supports the ROC\-based evaluations widely expected in regulatory pathways such as FDA device authorizationFood \([2007](https://arxiv.org/html/2608.10505#bib.bib218)\)and eases integration into rule\-based clinical decision systems\. Beyond radiology, conforming generative text to a perception model’s structured decisions applies to any domain where generated text must make decisions with tunable risk tolerance, such as autonomous systems, content moderation, and legal compliance\.

Limitations and risks\.RadFusion relies on the quality of all three components: classifier errors propagate into the rewritten report, the report generator may omit findings, and the LLM rewriter may introduce subtle linguistic artifacts\. The rewriting instruction is template\-based and tuned for chest X\-ray findings; our ablations show that its effectiveness varies across LLMs, so it would require re\-tuning for a different rewriter, imaging modality, or clinical domain\. Threshold control also operates over a predefined set of disease classes, and findings outside this set are not subject to threshold adjustment\. Finally, our evaluation relies on automatic label extraction from reports, which may itself introduce error\. We emphasize that our system is intended to assist, not replace, radiologist review\.

## References

- S\. Bannur, K\. Bouzid, D\. C\. Castro, A\. Schwaighofer, S\. Bond\-Taylor, M\. Ilse, F\. Pérez\-García, V\. Salvatelli, H\. Sharma, F\. Meissen,et al\.\(2024\)MAIRA\-2: Grounded Radiology Report Generation\.arXiv preprint arXiv:2406\.04449\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p1.1)\.
- When does optimizing a proper loss yield calibration?\.Advances in Neural Information Processing Systems36,pp\. 72071–72095\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p4.2)\.
- J\. M\. Z\. Chaves, S\. Huang, Y\. Xu, H\. Xu, N\. Usuyama, S\. Zhang, F\. Wang, Y\. Xie, M\. Khademi, Z\. Yang, H\. H\. Awadalla, J\. Gong, H\. Hu, J\. Yang, C\. Li, J\. Gao, Y\. Gu, C\. Wong, M\. Wei, T\. Naumann, M\. Chen, M\. P\. Lungren, S\. Yeung\-Levy, C\. P\. Langlotz, S\. Wang, and H\. Poon \(2024\)Towards a clinically accessible radiology foundation model: open\-access and lightweight, with automated evaluation\.External Links:[Link](https://api.semanticscholar.org/CorpusID:268379244)Cited by:[§1](https://arxiv.org/html/2608.10505#S1.p2.1),[§2](https://arxiv.org/html/2608.10505#S2.p1.1),[§4\.1](https://arxiv.org/html/2608.10505#S4.SS1.p1.1)\.
- Z\. Chen, Y\. Shen, Y\. Song, and X\. Wan \(2022\)Cross\-modal memory networks for radiology report generation\.arXiv preprint arXiv:2204\.13258\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p1.1)\.
- Z\. Chen, Y\. Song, T\. Chang, and X\. Wan \(2020\)Generating radiology reports via memory\-driven transformer\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing,Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p1.1)\.
- Z\. Chen, M\. Varma, J\. Delbrouck, M\. Paschali, L\. Blankemeier, D\. Van Veen, J\. M\. J\. Valanarasu, A\. Youssef, J\. P\. Cohen, E\. P\. Reis,et al\.\(2024\)Chexagent: towards a foundation model for chest x\-ray interpretation\.arXiv preprint arXiv:2401\.12208\.Cited by:[§1](https://arxiv.org/html/2608.10505#S1.p2.1),[§2](https://arxiv.org/html/2608.10505#S2.p1.1)\.
- N\. C\. F\. Codella, Y\. Jin, S\. Jain, Y\. Gu, H\. H\. Lee, A\. B\. Abacha, A\. Santamaria\-Pang, W\. Guyman, N\. Sangani, S\. Zhang, H\. Poon, S\. Hyland, S\. Bannur, J\. Alvarez\-Valle, X\. Li, J\. Garrett, A\. McMillan, G\. Rajguru, M\. Maddi, N\. Vijayrania, R\. Bhimai, N\. Mecklenburg, R\. Jain, D\. Holstein, N\. Gaur, V\. Aski, J\. Hwang, T\. Lin, I\. Tarapov, M\. Lungren, and M\. Wei \(2024\)MedImageInsight: an open\-source embedding model for general domain medical imaging\.External Links:2410\.06542,[Link](https://arxiv.org/abs/2410.06542)Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p1.1),[§3\.2](https://arxiv.org/html/2608.10505#S3.SS2.p1.1)\.
- S\. Dathathri, A\. Madotto, J\. Lan, J\. Hung, E\. Frank, P\. Molino, J\. Yosinski, and R\. Liu \(2019\)Plug and play language models: a simple approach to controlled text generation\.arXiv preprint arXiv:1912\.02164\.Cited by:[§1](https://arxiv.org/html/2608.10505#S1.p2.1),[§2](https://arxiv.org/html/2608.10505#S2.p3.1)\.
- M\. Ding, B\. Xiao, N\. Codella, P\. Luo, J\. Wang, and L\. Yuan \(2022\)DaViT: Dual attention vision transformers\.InEuropean conference on computer vision,pp\. 74–92\.Cited by:[§3\.2](https://arxiv.org/html/2608.10505#S3.SS2.p1.1)\.
- U\. Food \(2007\)Drug administration\. statistical guidance on reporting results from studies evaluating diagnostic tests\-guidance for industry and fda staff\.Food and Drug Administration, Center for Devices and Radiological Health Diagnostic Devices Branch, Division of Biostatistics, Office of Surveillance and Biometrics\.Cited by:[§1](https://arxiv.org/html/2608.10505#S1.p1.1),[§7](https://arxiv.org/html/2608.10505#S7.p1.1)\.
- C\. Fröhlich and R\. C\. Williamson \(2024\)Scoring rules and calibration for imprecise probabilities\.arXiv preprint arXiv:2410\.23001\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p4.2)\.
- C\. Guo, G\. Pleiss, Y\. Sun, and K\. Q\. Weinberger \(2017\)On calibration of modern neural networks\.InInternational Conference on Machine Learning,pp\. 1321–1330\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p4.2),[§3\.2](https://arxiv.org/html/2608.10505#S3.SS2.p3.1)\.
- X\. He, Y\. Zhang, L\. Mou, E\. Xing, and P\. Xie \(2020\)PathVQA: 30000\+ questions for medical visual question answering\.arXiv preprint arXiv:2003\.10286\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p2.1)\.
- J\. Ho and T\. Salimans \(2022\)Classifier\-free diffusion guidance\.arXiv preprint arXiv:2207\.12598\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p3.1)\.
- S\. L\. Hyland, S\. Bannur, K\. Bouzid, D\. C\. Castro, M\. Ranjit, A\. Schwaighofer, F\. Pérez\-García, V\. Salvatelli, S\. Srivastav, A\. Thieme,et al\.\(2023\)MAIRA\-1: a specialised large multimodal model for radiology report generation\.arXiv preprint arXiv:2311\.13668\.Cited by:[§1](https://arxiv.org/html/2608.10505#S1.p2.1),[§2](https://arxiv.org/html/2608.10505#S2.p1.1),[§4\.1](https://arxiv.org/html/2608.10505#S4.SS1.p1.1)\.
- J\. Irvin, P\. Rajpurkar, M\. Ko, Y\. Yu, S\. Ciurea\-Ilcus, C\. Chute, H\. Marklund, B\. Haghgoo, R\. Ball, K\. Shpanskaya,et al\.\(2019\)Chexpert: a large chest radiograph dataset with uncertainty labels and expert comparison\.InProceedings of the AAAI conference on artificial intelligence,Vol\.33,pp\. 590–597\.Cited by:[§3\.2](https://arxiv.org/html/2608.10505#S3.SS2.p1.1)\.
- S\. Jain, A\. Smit, S\. Q\. Truong, C\. D\. Nguyen, M\. Huynh, M\. Jain, V\. A\. Young, A\. Y\. Ng, M\. P\. Lungren, and P\. Rajpurkar \(2021\)VisualCheXbert: addressing the discrepancy between radiology report labels and image labels\.InProceedings of the Conference on Health, Inference, and Learning,pp\. 105–115\.Cited by:[§3\.2](https://arxiv.org/html/2608.10505#S3.SS2.p2.2),[§4\.2](https://arxiv.org/html/2608.10505#S4.SS2.p1.3)\.
- H\. Jin, H\. Che, Y\. Lin, and H\. Chen \(2024a\)Promptmrg: diagnosis\-driven prompts for medical report generation\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 2607–2615\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p1.1)\.
- \[19\]Y\. Jin, N\. C\. Codella, Y\. Xu, Y\. Gu, M\. Wei, H\. Fang, T\. Lin, P\. Vozila, and J\. HwangQRad: enhancing radiology report generation by captioning\-to\-vqa reframing\.InThe Second Workshop on GenAI for Health: Potential, Trust, and Policy Compliance,Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p1.1),[§3\.3](https://arxiv.org/html/2608.10505#S3.SS3.p2.1)\.
- Y\. Jin, Z\. Zhou, H\. Fang, and J\. Hwang \(2024b\)DAug: diffusion\-based channel augmentation for radiology image retrieval and classification\.InAdvancements In Medical Foundation Models: Explainability, Robustness, Security, and Beyond,Cited by:[§3\.2](https://arxiv.org/html/2608.10505#S3.SS2.p2.8)\.
- A\. E\. Johnson, T\. J\. Pollard, S\. J\. Berkowitz, N\. R\. Greenbaum, M\. P\. Lungren, C\. Deng, R\. G\. Mark, and S\. Horng \(2019\)MIMIC\-cxr, a de\-identified publicly available database of chest radiographs with free\-text reports\.Scientific data6\(1\),pp\. 317\.Cited by:[§1](https://arxiv.org/html/2608.10505#S1.p4.1),[§3\.2](https://arxiv.org/html/2608.10505#S3.SS2.p1.1),[§4\.1](https://arxiv.org/html/2608.10505#S4.SS1.p1.1)\.
- S\. Kadavath, T\. Conerly, A\. Askell, T\. Henighan, D\. Drain, E\. Perez, N\. Schiefer, Z\. Hatfield\-Dodds, N\. DasSarma, E\. Tran\-Johnson,et al\.\(2022\)Language models \(mostly\) know what they know\.arXiv preprint arXiv:2207\.05221\.Cited by:[§1](https://arxiv.org/html/2608.10505#S1.p2.1),[§2](https://arxiv.org/html/2608.10505#S2.p4.2),[§3\.2](https://arxiv.org/html/2608.10505#S3.SS2.p5.1)\.
- G\. K\. R\. Lau, H\. Dao, and B\. K\. H\. Low \(2025\)Uncertainty quantification for MLLMs\.InICLR Workshop: Quantify Uncertainty and Hallucination in Foundation Models: The Next Frontier in Reliable AI,External Links:[Link](https://openreview.net/forum?id=vCqtd8ksh3)Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p4.2)\.
- J\. J\. Lau, S\. Gayen, A\. Ben Abacha, and D\. Demner\-Fushman \(2018\)A dataset of clinically generated visual questions and answers about radiology images\.InScientific Data,Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p2.1)\.
- C\. Li, Z\. Li, Z\. Ge, and M\. Li \(2019\)Knowledge driven temporal activity localization\.Journal of Visual Communication and Image Representation64,pp\. 102628\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p1.1)\.
- C\. Li, C\. Wong, S\. Zhang, N\. Usuyama, H\. Liu, J\. Yang, T\. Naumann, H\. Poon, and J\. Gao \(2024\)LLava\-Med: training a Large Language\-and\-Vision Assistant for Biomedicine in One Day\.Advances in Neural Information Processing Systems36\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p2.1)\.
- Y\. Li, M\. Xiong, J\. Wu, and B\. Hooi \(2025\)Conftuner: training large language models to express their confidence verbally\.arXiv preprint arXiv:2508\.18847\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p4.2)\.
- Z\. Lin, S\. Guan, W\. Zhang, H\. Zhang, Y\. Li, and H\. Zhang \(2024\)Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models\.Artificial Intelligence Review\.Cited by:[§1](https://arxiv.org/html/2608.10505#S1.p1.1)\.
- B\. Liu, L\. Zhan, L\. Xu, L\. Ma, Y\. Yang, and X\. Wu \(2021\)SLAKE: a semantically\-labeled knowledge\-enhanced dataset for medical visual question answering\.arXiv preprint arXiv:2102\.09542\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p2.1)\.
- Q\. Liu, S\. Zhang, G\. Qin, Y\. Gu, Y\. Jin, S\. Preston, Y\. Xu, S\. Kiblawi, W\. Yim, T\. Ossowski,et al\.\(2026\)Scaling medical imaging report generation with multimodal reinforcement learning\.arXiv preprint arXiv:2601\.17151\.Cited by:[§3\.3](https://arxiv.org/html/2608.10505#S3.SS3.p3.1)\.
- P\. Manggala, A\. A\. Mastakouri, E\. Kirschbaum, S\. Kasiviswanathan, and A\. Ramdas \(2025\)QA\-calibration of language model confidence scores\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=D2hhkU5O48)Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p4.2)\.
- M\. Moor, Q\. Huang, S\. Wu, M\. Yasunaga, Y\. Dalmia, J\. Leskovec, C\. Zakka, E\. P\. Reis, and P\. Rajpurkar \(2023\)Med\-flamingo: a multimodal medical few\-shot learner\.InMachine Learning for Health, ML4H@NeurIPS 2023, 10 December 2023, New Orleans, Louisiana, USA,S\. Hegselmann, A\. Parziale, D\. Shanmugam, S\. Tang, M\. N\. Asiedu, S\. Chang, T\. Hartvigsen, and H\. Singh \(Eds\.\),Proceedings of Machine Learning Research, Vol\.225,pp\. 353–367\.External Links:[Link](https://proceedings.mlr.press/v225/moor23a.html)Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p2.1)\.
- A\. Nicolson, J\. Dowling, and B\. Koopman \(2023\)Improving chest x\-ray report generation by leveraging warm starting\.Artificial intelligence in medicine144,pp\. 102633\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p1.1)\.
- C\. Pellegrini, M\. Keicher, E\. Özsoy, and N\. Navab \(2023a\)Rad\-restruct: a novel vqa benchmark and method for structured radiology reporting\.InInternational Conference on Medical Image Computing and Computer\-Assisted Intervention,pp\. 409–419\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p2.1)\.
- C\. Pellegrini, E\. Özsoy, B\. Busam, N\. Navab, and M\. Keicher \(2023b\)Radialog: a large vision\-language model for radiology report generation and conversational assistance\.arXiv preprint arXiv:2311\.18681\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p2.1)\.
- J\. Platt \(1999\)Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods\.InAdvances in Large Margin Classifiers,Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p4.2)\.
- A\. Sellergren, S\. Kazemzadeh, T\. Jaroensri, A\. Kiraly, M\. Traverse, T\. Kohlberger, S\. Xu, F\. Jamil, C\. Hughes, C\. Lau,et al\.\(2025\)Medgemma technical report\.arXiv preprint arXiv:2507\.05201\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p1.1)\.
- A\. Singh, A\. Fry, A\. Perelman, A\. Tart, A\. Ganesh, A\. El\-Kishky, A\. McLaughlin, A\. Low, A\. Ostrow, A\. Ananthram,et al\.\(2025\)Openai gpt\-5 system card\.arXiv preprint arXiv:2601\.03267\.Cited by:[§3\.4](https://arxiv.org/html/2608.10505#S3.SS4.p1.3)\.
- L\. Sun, J\. J\. Zhao, W\. Han, and C\. Xiong \(2025\)Fact\-aware multimodal retrieval augmentation for accurate medical radiology report generation\.InProceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics,Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p1.1)\.
- T\. Tanida, P\. Müller, G\. Kaissis, and D\. Rueckert \(2023\)Interactive and explainable region\-guided radiology report generation\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 7433–7442\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p1.1)\.
- T\. Tu, S\. Azizi, D\. Driess, M\. Schaekermann, M\. Amin, P\. Chang, A\. Carroll, C\. Lau, R\. Tanno, I\. Ktena,et al\.\(2024\)Towards Generalist Biomedical AI\.NEJM AI1\(3\),pp\. AIoa2300138\.Cited by:[§1](https://arxiv.org/html/2608.10505#S1.p2.1),[§2](https://arxiv.org/html/2608.10505#S2.p1.1)\.
- X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou \(2023\)Self\-consistency improves chain of thought reasoning in language models\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p4.2)\.
- Y\. Wang, Z\. Lin, J\. He, M\. Zhang, L\. Wang, S\. Li, J\. Li, S\. K\. Zhou, and Z\. He \(2024\)Trust it or not: confidence\-guided automatic radiology report generation\.Neurocomputing\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p4.2)\.
- M\. Xiong, Z\. Hu, X\. Lu, Y\. Li, J\. Fu, J\. He, and B\. Hooi \(2024\)Can LLMs express their uncertainty? an empirical evaluation of confidence elicitation in LLMs\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2608.10505#S1.p2.1),[§2](https://arxiv.org/html/2608.10505#S2.p4.2)\.
- A\. Xu, B\. Lin, B\. Xue, B\. Wang, B\. Xu, B\. Wu, B\. Zhang, C\. Lin, C\. Dong, C\. Ling,et al\.\(2026\)Deepseek\-v4: towards highly efficient million\-token context intelligence\.arXiv preprint arXiv:2606\.19348\.Cited by:[Table 3](https://arxiv.org/html/2608.10505#S5.T3),[Table 3](https://arxiv.org/html/2608.10505#S5.T3.4.2)\.
- J\. Yang, C\. Li, P\. Zhang, B\. Xiao, C\. Liu, L\. Yuan, and J\. Gao \(2022\)Unified Contrastive Learning in Image\-Text\-Label Space\.External Links:2204\.03610Cited by:[§3\.2](https://arxiv.org/html/2608.10505#S3.SS2.p1.1)\.
- K\. Yang and D\. Klein \(2021\)FUDGE: controlled text generation with future discriminators\.InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,pp\. 3511–3535\.Cited by:[§1](https://arxiv.org/html/2608.10505#S1.p2.1),[§2](https://arxiv.org/html/2608.10505#S2.p3.1)\.
- X\. Zhang, Z\. Meng, J\. Lever, and E\. S\. Ho \(2025\)Libra: leveraging temporal images for biomedical radiology analysis\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 17275–17303\.Cited by:[§1](https://arxiv.org/html/2608.10505#S1.p2.1),[§2](https://arxiv.org/html/2608.10505#S2.p1.1)\.
- Y\. Zhang, X\. Wang, Z\. Xu, Q\. Yu, A\. Yuille, and D\. Xu \(2020\)When radiology report generation meets knowledge graph\.Proceedings of the AAAI Conference on Artificial Intelligence\.Cited by:[§2](https://arxiv.org/html/2608.10505#S2.p1.1)\.
- H\. Zhou, J\. N\. Acosta, S\. Adithan, S\. Datta, E\. J\. Topol, and P\. Rajpurkar \(2024\)MedVersa: a generalist foundation model for medical image interpretation\.arXiv preprint arXiv:2405\.07988\.Cited by:[§1](https://arxiv.org/html/2608.10505#S1.p2.1),[§2](https://arxiv.org/html/2608.10505#S2.p1.1)\.

## Appendix ASample Clinical User Interface

For additional context, we provide a sample user interface that demonstrates how RadFusion assists the daily radiology reporting process\. As shown in[Figure˜3](https://arxiv.org/html/2608.10505#A1.F3), the radiograph and an automatically generated report are presented to the user\. Alongside them, a set of threshold sliders for the disease classes is shown, each annotated with the model’s predicted confidence\. When the radiologist adjusts a slider, the corresponding operating point is highlighted on that class’s ROC curve, giving the user immediate context on the implied sensitivity–specificity trade\-off\. The adjusted report is displayed on the bottom right, where hovering the mouse over each text segment \(color\-underlined\) reveals the matching disease class, its predicted confidence, and the current threshold\. In this example, the user tuned down the threshold for Edema, which turns the class positive \(third sentence, orange underlined\) in the adjusted report\.

![Refer to caption](https://arxiv.org/html/2608.10505v1/figs/sample_ui.png)Figure 3:A sample clinical user interface for RadFusion\. The radiograph and the automatically generated report are shown to the user, together with per\-class threshold sliders annotated with the model’s predicted confidence\. Adjusting a slider highlights the corresponding operating point on the class’s ROC curve, conveying the implied sensitivity–specificity trade\-off\. The adjusted report \(bottom right\) links each color\-underlined text segment to its disease class, predicted confidence, and threshold on hover\. Here, lowering the Edema threshold turns the class positive \(third sentence, orange underlined\)
## Appendix BThreshold\-Control Evaluation on All Disease Classes

[Figure˜4](https://arxiv.org/html/2608.10505#A2.F4)presents the full threshold\-control evaluation on all 13 disease classes, extending the four representative classes highlighted in the main text \([Figure˜2](https://arxiv.org/html/2608.10505#S4.F2)\)\. For each class, we compare the classifier ROC \(blue\), computed directly from the classifier’s continuous confidence scores, against the ROC of the threshold\-controlled reports \(red\), obtained by sweepingτ\\taufrom 0\.0 to 1\.0 and converting each rewritten report back to class labels, as described in[Section˜4\.2](https://arxiv.org/html/2608.10505#S4.SS2)\. The green cross marks the single operating point of the non\-controlled report\. Across all classes, the report ROC closely tracks the classifier ROC, and the non\-controlled report falls below the curves, confirming that the conformance and factual\-correctness improvements reported in[Section˜4](https://arxiv.org/html/2608.10505#S4)hold consistently across the full label set rather than only the highlighted classes\.

![Refer to caption](https://arxiv.org/html/2608.10505v1/x2.png)Figure 4:Threshold\-controllable report generation on all 13 disease classes\. High overlap between classifier ROC \(blue\) and report ROC \(red\) confirms conformance\. The non\-controlled report \(green cross\) falls below the curves\. Ground\-truth labels are obtained from the reference reports via VisualCheXbert\.
## Appendix CComparison of Classifier Implementations

[Figure˜5](https://arxiv.org/html/2608.10505#A3.F5)shows the per\-class ROC curves of the two strongest classifier implementations compared in[Section˜5\.1](https://arxiv.org/html/2608.10505#S5.SS1): the fine\-tuned MI2 classifier \(MI2 FT\) and a linear probe ofQRad’s frozen encoder \(QRad Linear Prob\)\. These curves provide the full operating\-point view underlying the summary AUC\-ROC reported in[Table˜1](https://arxiv.org/html/2608.10505#S5.T1)\. Consistent with the table, the two track each other closely across most classes, withQRad Linear Prob pulling ahead on hard classes such as Pneumothorax and MI2 FT stronger on visually obvious classes such as Support Devices\.

![Refer to caption](https://arxiv.org/html/2608.10505v1/x3.png)Figure 5:Per\-class ROC curves of the fine\-tuned MI2 classifier \(MI2 FT\) and a linear probe ofQRad’s frozen encoder \(QRad Linear Prob\) across the 13 disease classes\. The two perform comparably across most classes, withQRad Linear Prob stronger on hard classes such as Pneumothorax and MI2 FT stronger on visually obvious classes such as Support Devices\.
## Appendix DThreshold\-Control Evaluation with the Alternative Classifier

The threshold\-control evaluation in[Appendix˜B](https://arxiv.org/html/2608.10505#A2)uses the default MI2 FT classifier as the perception model\. Here we repeat it with theQRad Linear Prob classifier \([Appendix˜C](https://arxiv.org/html/2608.10505#A3)\), an alternative implementation that shares its encoder with the report generator\.[Figure˜6](https://arxiv.org/html/2608.10505#A4.F6)shows the per\-class threshold\-control evaluation across all 13 disease classes, with GPT\-5 as the rewriter\. As with the default classifier, the report ROC \(red\) closely tracks the classifier ROC \(blue\), and the non\-controlled report \(green cross\) falls below the curves, confirming that RadFusion’s threshold conformance holds across the underlying classifier\.

![Refer to caption](https://arxiv.org/html/2608.10505v1/x4.png)Figure 6:Threshold\-control evaluation on all 13 disease classes using theQRad Linear Prob classifier as the perception model\. As with the default MI2 FT classifier, the report ROC \(red\) closely tracks the classifier ROC \(blue\), and the non\-controlled report \(green cross\) falls below the curves\. Ground\-truth labels are obtained from the reference reports via VisualCheXbert\.
## Appendix ELLM Rewriting Instruction

[Figure˜7](https://arxiv.org/html/2608.10505#A5.F7)shows the full instruction prompt provided to the LLM rewriter\. The prompt consists of three parts: \(1\) a task description specifying the rewriting objective, \(2\) a template dictionary providing example positive and negative text segments for each of the 14 CheXbert disease classes, and \(3\) worked examples illustrating the rewriting rules for different cases\. At inference, the placeholder\{INPUT\}is replaced with the original report and the threshold\-adjusted classification results\.

System Instruction:

You are given an input chest x\-ray radiology report and the classification result of 14 CheXbert classes, which includes both positive and negative diseases\.

Modify the report so that it matches each class in the CheXbert classification results\. Add text segments or sentences to the report if the class is not mentioned in the report, or change/remove sentences in the report so that it agrees with the classification\.

Your modified report should be a valid Findings section of Chest X\-ray radiology report, with information like the location, severity of medical findings based on the original report\.

When overwriting the report with the positive and negative classes, consider using text segments from the following templates, and fit them into the report as appropriate\.

Template Dictionary\(abbreviated; full version includes all 14 classes\):

\{

"Cardiomegaly": \{

"positive": \["The heart is moderately enlarged\.",

"Mild enlargement of the cardiac silhouette\.", \.\.\.\],

"negative": \["The cardiac silhouette is normal\.",

"The heart is normal in size\.", \.\.\.\]

\},

"Pneumothorax": \{

"positive": \["small apical pneumothorax", \.\.\.\],

"negative": \["no pneumothorax", \.\.\.\]

\}, \.\.\.

\}

Rewriting Rules:

Example 1:If "Fracture" is a positive class:

\-\- If the original report indicates Fracture is positive, do nothing\.

\-\- If the original report indicates Fracture is negative, replace that sentence with a positive text segment\.

\-\- If the original report doesn’t mention Fracture, choose a positive text segment and add it to the report\.

Example 2:If "Support Devices" is a negative class:

\-\- If the original report doesn’t indicate the existence of any support device, do nothing\.

\-\- If the original report mentions any kind of support device exists, remove it\.

The input and output report are the Findings section of a radiology report\. Make sure all positive classes are identifiable in the modified report\. Return the modified version as JSON with "output" as the key\.

Only return the JSON without any explanation\. Double check and make sure you didn’t forget any classes in the lists, and didn’t make unnecessary changes:

\{INPUT\}

Figure 7:The instruction prompt for LLM\-based report rewriting\. The LLM receives the original report, threshold\-adjusted classification results, and a template dictionary of example text segments for 14 CheXbert disease classes\. The full template dictionary contains positive and negative text segments for all classes: Enlarged Cardiomediastinum, Cardiomegaly, Lung Opacity, Lung Lesion, Edema, Consolidation, Pneumonia, Atelectasis, Pneumothorax, Pleural Effusion, Pleural Other, Fracture, and Support Devices\.