CoMedBench: A Multi-Source Benchmark of Synthetic Medical Data Fidelity and Downstream Utility

# CoMedBench: A Multi-Source Benchmark of Synthetic Medical Data Fidelity and Downstream Utility
Source: [https://arxiv.org/html/2608.12805](https://arxiv.org/html/2608.12805)
Al Amin FarhadMrinmoy Sarkar AntoDavid RehkopfAyin ValaTanmoy Sarkar Pias

###### Abstract

Access to clinical data is essential for developing reliable healthcare machine learning systems, but direct use of electronic health records is constrained by privacy regulation, institutional review, data\-use agreements, and the risk of re\-identification\. Synthetic data promises a practical alternative: it can preserve useful statistical and clinical structure while reducing exposure of sensitive patient records\. Yet the evidence for synthetic data utility in healthcare remains fragmented\. Prior studies often evaluate a single generator, one dataset, or a narrow downstream task, making it difficult to know when synthetic data can support model development and when it fails to preserve task\-critical signal\. We introduceCoMedBench, a reproducible benchmark that evaluates a family of generators under a common clinical\-validity framework and one shared training and evaluation engine, spanning static tabular and temporal downstream tasks on established critical\-care datasets\. In total the benchmark spans 37 dataset\-task pairs across two modalities consists of 20 static tabular and 17 temporal ICU time\-series\-drawn from seven public data sources: three intensive\-care databases \(MIMIC\-III, MIMIC\-IV, and eICU\) together with theUCIMachine Learning Repository, the CDCBRFSSdiabetes cohort \(2015\),NHANES\(1999\-2014\), and the pycox survival datasets \(GBSGandMETABRIC\)\. The benchmark evaluates both statistical fidelity and task utility by comparing models trained and tested across real and synthetic data\. In these settings, synthetic training data preserves most of the downstream signal: on tabular tasks the reference generator CoMed\-CTGAN retains a mean AUROC utility \(the synthetic\-to\-real performance ratio\) of90\.6%90\.6\\%, rising to97\.3%97\.3\\%for the strongest generator, CoMed\-TVAE\. Temporal ICU tasks are harder and more generator\-sensitive: CoMed\-CTGAN retains81\.6%81\.6\\%\(AUROC\) and only64\.0%64\.0\\%under the imbalance\-sensitive AUPRC, whereas CoMed\-TVAE still retains∼95%\{\\sim\}95\\%\(AUROC\)\. By organizing evaluation around downstream clinical prediction rather than visual similarity alone, the benchmark clarifies where synthetic data is useful, where temporal structure remains a bottleneck, and how future generators should be assessed for healthcare AI\.

## Introduction

![Refer to caption](https://arxiv.org/html/2608.12805v1/figures/INTRO_tsne_v2.png)Figure 1:Two\-dimensional t\-SNE \(perplexity3030, standardized features\) of real vs\. synthetic ICU records for the MIMIC\-III heart\-failure cohort\. Generators differ markedly in how well synthetic data covers the real manifold: higher\-fidelity models \(CoMed\-TVAE, CoMed\-GC\) mix throughout the real points, while lower\-fidelity models \(CoMed\-CTGAN, CoMed\-CopulaGAN\) leave real regions under\-covered\. This gap between visual/statistical similarity and task usefulness motivates our benchmark\.Modern healthcare AI depends on large, representative, and carefully curated clinical datasets\([8](https://arxiv.org/html/2608.12805#bib.bib12)\)\. Electronic health records \(EHRs\), intensive\-care databases, laboratory measurements, and longitudinal monitoring streams contain signals that support mortality prediction, readmission modeling, length\-of\-stay estimation, and phenotyping\. However, the same data are hard to share: clinical datasets include protected health information, rare diagnoses, and combinations of demographic and clinical attributes that can make patients identifiable even after conventional de\-identification\. As a result, many healthcare machine learning studies depend on a small number of public resources such as MIMIC\-III, MIMIC\-IV, and eICU, while large institutional datasets remain inaccessible to most researchers\([11](https://arxiv.org/html/2608.12805#bib.bib1);[10](https://arxiv.org/html/2608.12805#bib.bib2);[22](https://arxiv.org/html/2608.12805#bib.bib3)\)\.

Figure 2:Benchmark coverage\. The 37 dataset\-task pairs split by modality \(tabular/public vs\. time\-series/ICU\) and by source; the public UCI datasets are further grouped into clinical subclasses \(mortality, diabetes, cancer, and other diseases\), while each ICU database \(MIMIC\-III, MIMIC\-IV, eICU\) contributes mortality, readmission, and length\-of\-stay tasks\.Synthetic clinical data has emerged as a promising way to reduce this access barrier\. Instead of releasing original patient records, a data holder trains a generative model and releases artificial records intended to approximate the distribution of the source data\. If successful, synthetic data can enable algorithm prototyping, external benchmarking, software testing, and multi\-institutional collaboration without exposing raw records\.

The central challenge is that synthetic data must be useful, not merely realistic\. A generator can match marginal distributions while distorting rare outcomes, comorbidity structure, medication\-laboratory dependencies, or temporal dynamics\. These distortions are harmful for downstream tasks because the predictive signal often lies in minority classes, rare events, and time\-dependent trajectories\([20](https://arxiv.org/html/2608.12805#bib.bib11)\)\. A synthetic dataset that appears faithful under aggregate statistics may therefore fail when used to train a model for a clinically relevant task; Figure[1](https://arxiv.org/html/2608.12805#Sx1.F1)illustrates this gap\.

Existing synthetic\-data evaluations in healthcare report fidelity, privacy, and utility, but utility is often measured under different datasets, models, metrics, and task definitions, which makes comparison difficult\. Researchers thus still lack a practical answer to a basic question: for which healthcare downstream tasks can synthetic data act as a useful substitute for real clinical data?

This paper addresses that gap with CoMedBench, a benchmark built to make synthetic\-data evaluation controlled and comparable\. CoMedBench applies a model\-agnostic clinical\-validity layer and wraps a family of generators in a single training pipeline so that the generator is the only component that varies between runs, and evaluates each synthetic table through two complementary scenarios: statistical fidelity, which measures how closely generated records resemble the source distribution, and downstream utility, which measures how well models trained on synthetic data transfer to real test data\. This design separates apparent realism from practical usefulness\.

Our work makes the following contributions:

- •Controlled evaluation under domain validity\.We evaluate four generator families under a common clinical\-validity framework and one shared training and evaluation pipeline, so performance is compared without confounding differences in preprocessing, structural validation, or downstream evaluation\.
- •A multi\-source clinical benchmark\.37 dataset\-task pairs\-20 static tabular and 17 temporal ICU time\-series from seven public sources, each evaluated on statistical fidelity and downstream utility\.
- •Synthetic data preserves downstream signal, but unevenly\.The strongest generator, CoMed\-TVAE, retains∼97%\{\\sim\}97\\%of real\-data AUROC on tabular tasks and∼95%\{\\sim\}95\\%on time\-series; utility is high on static tabular tasks but degrades on temporal, imbalanced ICU tasks \(AUPRC utility as low as∼64%\{\\sim\}64\\%\), and no single generator dominates across the evaluation settings\.
- •Fidelity is a necessary but insufficient proxy\.Overall statistical fidelity correlates with downstream utility only moderately \(Overallr=0\.67r=0\.67,ρ=0\.74\\rho=0\.74\); utility must therefore be measured directly, with rare\-event temporal ICU tasks the main bottleneck\.

## Related Works

Figure 3:The CoMedBench pipeline\. Real data \(used directly as a flat table, or an ICU stay’s multivariate series summarized into a per\-stay feature vector\) drives a single configuration\-driven engine\-preprocessing and metadata, clinical\-validity controls, and a synthesizerGθG\_\{\\theta\}\-whose only stage that changes across runs is the synthesizer\. The resulting synthetic tableDsynD\_\{\\mathrm\{syn\}\}and the real tableDrealD\_\{\\mathrm\{real\}\}are then compared by training the same five classifiers on each and testing on an*identical*held\-out real set \(TRTR vs\. TSTR\), yielding the utility ratio\.### Synthetic Data in Tabular and Time\-Series Data

Synthetic data for structured data can be grouped into single\-table, multi\-table, and time\-series synthesis\. Single\-table methods generate one flat table with mixed categorical and continuous columns, focusing on distributional fidelity, privacy, and machine\-learning utility; examples include CTGAN\-style models, differentially private normalizing flows, diffusion models, and autoencoder\-based tabular generators\([29](https://arxiv.org/html/2608.12805#bib.bib4);[17](https://arxiv.org/html/2608.12805#bib.bib5);[14](https://arxiv.org/html/2608.12805#bib.bib9)\)\. These methods suit static clinical cohorts but may lose key relationships when a relational EHR database is flattened into one table\.

Multi\-table synthesis instead preserves dependencies across interconnected tables \(e\.g\., patient\-admission, diagnosis\-procedure, and laboratory\-medication relationships\), better reflecting real\-world healthcare databases in which downstream labels depend on information distributed across multiple tables\. Such relational methods are important for reconstructing realistic EHR structure rather than only generating independent rows\.

Time\-series synthesis focuses on sequential records where order, missingness, irregular sampling, and temporal dynamics are clinically meaningful\. This direction is especially relevant for ICU tasks because prior work using models such as GRU\-D and established MIMIC\-III benchmarks has shown that temporal trends and missingness patterns are predictive for mortality, decompensation, length of stay, and phenotyping\([3](https://arxiv.org/html/2608.12805#bib.bib6);[9](https://arxiv.org/html/2608.12805#bib.bib7)\)\.

### Synthetic Data in Downstream Tasks

Synthetic data is most useful when it supports downstream tasks, not only when it looks statistically similar to real data\. A common evaluation is train\-on\-synthetic, test\-on\-real \(TSTR\): models trained on generated data are evaluated on a held\-out real test set, directly measuring whether synthetic data preserves transferable predictive signal\.

Established MIMIC\-III and MIMIC\-IV benchmarks provide common structured and time\-series tasks\-mortality, readmission, length\-of\-stay, and phenotyping\-for evaluating whether synthetic clinical data preserves useful predictive signal\([9](https://arxiv.org/html/2608.12805#bib.bib7);[3](https://arxiv.org/html/2608.12805#bib.bib6);[6](https://arxiv.org/html/2608.12805#bib.bib8);[7](https://arxiv.org/html/2608.12805#bib.bib27);[2](https://arxiv.org/html/2608.12805#bib.bib24);[15](https://arxiv.org/html/2608.12805#bib.bib14);[19](https://arxiv.org/html/2608.12805#bib.bib23);[23](https://arxiv.org/html/2608.12805#bib.bib22);[16](https://arxiv.org/html/2608.12805#bib.bib26);[1](https://arxiv.org/html/2608.12805#bib.bib13);[12](https://arxiv.org/html/2608.12805#bib.bib25)\)\. These tasks motivate our benchmark because synthetic data may perform well on static tabular prediction while failing to preserve temporal patterns or rare\-event signals\.

### Synthetic Data in Healthcare AI

Public datasets such as MIMIC\-III, MIMIC\-IV, and eICU have enabled critical\-care research, but broad access to institutional EHR data remains limited\([11](https://arxiv.org/html/2608.12805#bib.bib1);[10](https://arxiv.org/html/2608.12805#bib.bib2);[22](https://arxiv.org/html/2608.12805#bib.bib3)\)\.

Synthetic healthcare data can support prototyping, testing, education, and benchmarking without exposing raw patient records\([4](https://arxiv.org/html/2608.12805#bib.bib29);[27](https://arxiv.org/html/2608.12805#bib.bib28)\)\. However, privacy protection alone is insufficient: generated data must also preserve clinical structure such as outcome prevalence, comorbidity patterns, medication\-laboratory relationships, and temporal deterioration\. This motivates our benchmark, which evaluates whether synthetic data supports established downstream tasks rather than only matching broad statistical properties\.

## Methodology

### Overview of the CoMedBench Framework

CoMedBench treats synthetic clinical\-data generation and evaluation as a single, configuration\-driven procedure applied uniformly to every \(dataset×\\timestask\) pair\. Each experiment is defined by a compact task specification: the input feature matrix, the primary\-key column, the prediction label and the feature\-typing rule\. One shared engine consumes this specification and executes four stages \(Figure[3](https://arxiv.org/html/2608.12805#Sx2.F3)\): \(1\)preprocessing and metadata, which loads the flat feature matrix, imputes missing values, and assigns each column an explicit type; \(2\)clinical\-validity controls, a model\-agnostic layer that enforces schema and task\-specific validity rules consistently across every generator; \(3\)synthesis, which fits a generatorGθG\_\{\\theta\}and samples a synthetic tableDsynD\_\{\\mathrm\{syn\}\}whose label distribution matches the real class proportions; and \(4\)evaluation, which scoresDsynD\_\{\\mathrm\{syn\}\}against the real dataDrealD\_\{\\mathrm\{real\}\}along two axes, statistical*fidelity*and downstream machine\-learning*utility*\(TRTR vs\. TSTR\)\. Because the same four stages are applied to every dataset, task, and generator, only the synthesis stage \(or a single ablated setting\) changes between runs, making the study a controlled benchmark rather than a comparison of loosely coupled implementations\.

### Datasets, Tasks, and Representation

We evaluate on three intensive\-care databases \- MIMIC\-III, MIMIC\-IV, and eICU\([11](https://arxiv.org/html/2608.12805#bib.bib1);[10](https://arxiv.org/html/2608.12805#bib.bib2);[22](https://arxiv.org/html/2608.12805#bib.bib3)\), together with a collection of public clinical and tabular datasets\([25](https://arxiv.org/html/2608.12805#bib.bib18);[18](https://arxiv.org/html/2608.12805#bib.bib20);[24](https://arxiv.org/html/2608.12805#bib.bib19);[5](https://arxiv.org/html/2608.12805#bib.bib16);[13](https://arxiv.org/html/2608.12805#bib.bib21);[28](https://arxiv.org/html/2608.12805#bib.bib17);[21](https://arxiv.org/html/2608.12805#bib.bib10)\)\. Tasks are binary or multi\-class prediction: in\-hospital mortality, ICU and hospital 30\-day readmission, ICU length of stay, and disease/outcome classification\. Figure[2](https://arxiv.org/html/2608.12805#Sx1.F2)shows how the suite breaks down by modality, source, and task\.

Every task is modeled as a single flat table \(one row per patient or stay\)\. In the*tabular*representation the dataset is already a flat feature matrix and is used directly\. In the*time\-series\-to\-tabular*representation, each ICU stay’s multivariate time series is summarized into a per\-stay feature vector: each clinical variablevvis represented byvfirstv\_\{\\mathrm\{first\}\},vminv\_\{\\min\},vmaxv\_\{\\max\}, andvmeanv\_\{\\mathrm\{mean\}\}\(and, where available,vlastv\_\{\\mathrm\{last\}\},vmedianv\_\{\\mathrm\{median\}\},vstdv\_\{\\mathrm\{std\}\}\) alongside demographic and contextual features, yielding one fixed\-width row per stay\.

### Preprocessing and Metadata

Each table is loaded and assigned a primary key, a native identifier where one exists \(e\.g\., stay id\) or a surrogate key otherwise; rows with a missing label are dropped\. We build single\-table metadata assigning every column an explicit type: the key as an identifier \(regenerated at sampling time\), the label as categorical, and each feature as categorical or numerical\. Missing values follow a domain\-aware imputation policy, and low\-signal columns are pruned\([26](https://arxiv.org/html/2608.12805#bib.bib15)\)\. High\-cardinality categorical fields are encoded so that generators emit only categories observed in training\.

### Clinical Validity Controls

To ensure that generated records satisfy basic clinical and structural validity requirements, we apply a model\-agnostic domain\-validity layer consistently across all evaluated generators\. The layer uses schema information and task\-specific validity rules to prevent clinically impossible or structurally invalid outputs\. This study evaluates the effect of applying the same validity layer across generator families\.

### Synthesis: Generator Families

The same clinical\-validity layer is applied identically to four single\-table generators from distinct families, so the synthesis stage is the only component that changes between runs\.CoMed\-CTGANis a conditional tabular generative adversarial network with mode\-specific normalization and training\-by\-sampling for imbalanced discrete columns\([29](https://arxiv.org/html/2608.12805#bib.bib4)\)\.CoMed\-TVAEis a tabular variational autoencoder,CoMed\-CopulaGANa copula\-augmented conditional GAN, andCoMed\-GaussianCopulaa statistical Gaussian\-copula model with no training epochs\. The GAN\-based and TVAE\-based generators use standard published configurations\. Because the labels are imbalanced, generation is*label\-conditioned*: the synthetic label distribution matches the real class proportions, and the number of synthetic rows equals the real dataset size\.

### Evaluation

We evaluate every synthetic table against the real table on two axes: statistical fidelity and downstream machine\-learning utility \(TRTR/TSTR\)\. For brevity writeR=DrealR=D\_\{\\mathrm\{real\}\}andS=DsynS=D\_\{\\mathrm\{syn\}\}, and let𝒞\\mathcal\{C\}be the set of modeled columns, partitioned into numerical columns𝒞num\\mathcal\{C\}\_\{\\mathrm\{num\}\}and categorical columns𝒞cat\\mathcal\{C\}\_\{\\mathrm\{cat\}\}, with𝒫\\mathcal\{P\}the set of column pairs\.

Figure 4:The CoMedBench fidelity suite\. A*Quality*score combines Distribution Stability \(per\-column marginals\) and Correlation Stability \(column\-pair trends\); a*Diagnostic*score adds Data Validity and Data Structure\. All scores lie in\[0,1\]\[0,1\]\(1=1=identical\)\.#### Statistical fidelity\.

Figure[4](https://arxiv.org/html/2608.12805#Sx3.F4)organizes the fidelity metrics used below; each is computed by CoMedBench from standard column and column\-pair statistics\.*Distribution Stability*averages a per\-column marginal score: the Kolmogorov\-Smirnov complement for numerical columns and the total\-variation complement for categorical columns\. For a numerical columnccwith empirical CDFsF^cR,F^cS\\hat\{F\}^\{R\}\_\{c\},\\hat\{F\}^\{S\}\_\{c\},

KSC⁡\(c\)=1−supx\|F^cR​\(x\)−F^cS​\(x\)\|,\\mathrm\{KSC\}\(c\)=1\-\\sup\_\{x\}\\bigl\|\\hat\{F\}^\{R\}\_\{c\}\(x\)\-\\hat\{F\}^\{S\}\_\{c\}\(x\)\\bigr\|,\(1\)and for a categorical columnccwith category set𝒜c\\mathcal\{A\}\_\{c\}and relative frequenciesRc​\(a\),Sc​\(a\)R\_\{c\}\(a\),S\_\{c\}\(a\),

TVC⁡\(c\)=1−12​∑a∈𝒜c\|Rc​\(a\)−Sc​\(a\)\|\.\\mathrm\{TVC\}\(c\)=1\-\\tfrac\{1\}\{2\}\\sum\_\{a\\in\\mathcal\{A\}\_\{c\}\}\\bigl\|R\_\{c\}\(a\)\-S\_\{c\}\(a\)\\bigr\|\.\(2\)Distribution Stability is their mean over columns,

Shapes=1\|𝒞\|​\(∑c∈𝒞numKSC⁡\(c\)\+∑c∈𝒞catTVC⁡\(c\)\)\.\\mathrm\{Shapes\}=\\frac\{1\}\{\|\\mathcal\{C\}\|\}\\\!\\left\(\\sum\_\{c\\in\\mathcal\{C\}\_\{\\mathrm\{num\}\}\}\\\!\\mathrm\{KSC\}\(c\)\+\\sum\_\{c\\in\\mathcal\{C\}\_\{\\mathrm\{cat\}\}\}\\\!\\mathrm\{TVC\}\(c\)\\right\)\.\(3\)*Correlation Stability*averages a per\-pair dependency score\. For a numerical pair\(a,b\)\(a,b\)with Pearson correlationsρa​bR,ρa​bS\\rho^\{R\}\_\{ab\},\\rho^\{S\}\_\{ab\},

CS⁡\(a,b\)=1−\|ρa​bR−ρa​bS\|2,\\mathrm\{CS\}\(a,b\)=1\-\\frac\{\\bigl\|\\rho^\{R\}\_\{ab\}\-\\rho^\{S\}\_\{ab\}\\bigr\|\}\{2\},\(4\)and for any pair involving a categorical column \(numerical columns discretized into bins\) with joint relative frequenciesRa​b​\(i,j\),Sa​b​\(i,j\)R\_\{ab\}\(i,j\),S\_\{ab\}\(i,j\),

CT⁡\(a,b\)=1−12​∑i,j\|Ra​b​\(i,j\)−Sa​b​\(i,j\)\|\.\\mathrm\{CT\}\(a,b\)=1\-\\tfrac\{1\}\{2\}\\sum\_\{i,j\}\\bigl\|R\_\{ab\}\(i,j\)\-S\_\{ab\}\(i,j\)\\bigr\|\.\(5\)Correlation Stability is

Trends=1\|𝒫\|​∑\(a,b\)∈𝒫m⁡\(a,b\),\\mathrm\{Trends\}=\\frac\{1\}\{\|\\mathcal\{P\}\|\}\\sum\_\{\(a,b\)\\in\\mathcal\{P\}\}m\(a,b\),\(6\)where

m⁡\(a,b\)=\{CS⁡\(a,b\),a,b∈𝒞num,CT⁡\(a,b\),otherwise,m\(a,b\)=\\begin\{cases\}\\mathrm\{CS\}\(a,b\),&a,b\\in\\mathcal\{C\}\_\{\\mathrm\{num\}\},\\\\\[2\.0pt\] \\mathrm\{CT\}\(a,b\),&\\text\{otherwise,\}\\end\{cases\}\(7\)and the*Overall*quality score is the mean of the two properties,

Overall=12​\(Shapes\+Trends\)\.\\mathrm\{Overall\}=\\tfrac\{1\}\{2\}\\bigl\(\\mathrm\{Shapes\}\+\\mathrm\{Trends\}\\bigr\)\.\(8\)The diagnostic report additionally measures*Data Validity*and*Data Structure*\. Data Validity averages a per\-column adherence score: boundary adherence for numerical columns and category adherence for categorical columns,

BA⁡\(c\)\\displaystyle\\mathrm\{BA\}\(c\)=1\|S\|∑s∈Sc\[mincR≤s≤maxcR\],\\displaystyle=\\frac\{1\}\{\|S\|\}\\sum\_\{s\\in S\_\{c\}\}\\mathbf\{1\}\\\!\\left\[\\min\_\{c\}R\\leq s\\leq\\max\_\{c\}R\\right\],\(9\)CA⁡\(c\)\\displaystyle\\mathrm\{CA\}\(c\)=1\|S\|∑s∈Sc\[s∈𝒜cR\],\\displaystyle=\\frac\{1\}\{\|S\|\}\\sum\_\{s\\in S\_\{c\}\}\\mathbf\{1\}\\\!\\left\[s\\in\\mathcal\{A\}^\{R\}\_\{c\}\\right\],\(10\)with primary keys scored by uniquenessKU=1−1\|S\|​\#​\{missing or duplicate synthetic keys\}\\mathrm\{KU\}=1\-\\tfrac\{1\}\{\|S\|\}\\\#\\\{\\text\{missing or duplicate synthetic keys\}\\\}\. Data Structure is the Jaccard overlap of the \(column name, dtype\) schemas𝒦R,𝒦S\\mathcal\{K\}^\{R\},\\mathcal\{K\}^\{S\},

Structure=\|𝒦R∩𝒦S\|\|𝒦R∪𝒦S\|\.\\mathrm\{Structure\}=\\frac\{\|\\mathcal\{K\}^\{R\}\\cap\\mathcal\{K\}^\{S\}\|\}\{\|\\mathcal\{K\}^\{R\}\\cup\\mathcal\{K\}^\{S\}\|\}\.\(11\)All fidelity scores lie in\[0,1\]\[0,1\]\(11= identical\)\.

#### Machine\-learning utility \(TRTR/TSTR\)\.

For each task we train five classifiers\- Logistic Regression, Random Forest, Gradient Boosting, XGBoost, and an MLP, under two regimes:*TRTR*\(Train Real, Test Real; upper\-bound reference\) and*TSTR*\(Train Synthetic, Test Real, on the*same*held\-out real test set\)\. We summarize the gap as a Utility ratio,

Utility%=AUCTSTRAUCTRTR×100,\\mathrm\{Utility\\%\}=\\frac\{\\mathrm\{AUC\}\_\{\\mathrm\{TSTR\}\}\}\{\\mathrm\{AUC\}\_\{\\mathrm\{TRTR\}\}\}\\times 100,\(12\)so100%100\\%means synthetic\-trained models match real\-trained ones\.

## Results

We report statistical fidelity across the four generators, downstream utility, and the relationship between them\. Fidelity is measured for all four synthesizers, with a per\-dataset breakdown in Table[2](https://arxiv.org/html/2608.12805#Sx4.T2)\. Downstream utility is reported per dataset for all four synthesizers in Table[3](https://arxiv.org/html/2608.12805#Sx4.T3)\(AUROC and AUPRC\), each aggregated over the five classifiers\.

Table 1:Effect of the clinical\-validity layer on statistical fidelity\. Each cell reports Overall fidelity with the validity layer, and in parentheses the change relative to the same generator without it \(positive==the validity layer improves fidelity\)\. The layer improves fidelity for the GAN\-based CoMed\-CTGAN and CoMed\-CopulaGAN on nearly every task \(up to\+0\.098\+0\.098\), while the CoMed\-TVAE and CoMed\-GaussianCopula models change only marginally \(\|Δ\|≤0\.04\|\\Delta\|\\leq 0\.04; slightly negative for CoMed\-TVAE\)\.TabularDatasetTaskCoMedCTGANCoMedTVAECoMedCGCoMedGCBreast Cancer CoimbraBreast Cancer0\.7440\.8930\.7600\.895CDC BRFSSDiabetes0\.9700\.9400\.9400\.849GBSG \(pycox\)Breast Cancer Survival0\.9350\.9670\.9090\.883METABRIC \(pycox\)Breast Cancer Survival0\.9530\.9790\.9290\.907SUPPORT2Death0\.8280\.8660\.8250\.867SUPPORT2Functional Outcome0\.8160\.8790\.8470\.823SUPPORT2Mortality0\.8460\.8730\.8100\.863UCI Cervical CancerCervical Cancer0\.8670\.9130\.8280\.877UCI Kidney DiseaseChronic Kidney Disease0\.8460\.8700\.8380\.819UCI DermatologyDermatological Disease0\.9050\.9660\.8990\.841UCI Diabetes30\-day Readmission0\.7940\.7430\.7880\.825UCI DiabetesAny\-Readmission0\.7950\.7650\.7860\.827UCI DiabetesReadmission \(3\-class\)0\.7850\.7650\.7630\.824UCI Early DiabetesEarly Diabetes Risk0\.8860\.9580\.8630\.922UCI Heart FailureHeart Failure Mortality0\.8800\.9290\.8350\.922UCI HepatitisHepatitis Mortality0\.8960\.9400\.8840\.830UCI Indian LiverLiver Disease0\.8300\.9360\.8550\.918UCI MammographicMass Malignancy0\.9510\.9770\.9400\.839UCI Statlog HeartHeart Disease0\.9030\.9460\.8630\.829NHANESMortality0\.7670\.9440\.8040\.944
Time\-seriesDatasetTaskCoMedCTGANCoMedTVAECoMedCGCoMedGCMIMIC\-IIICOPD Mortality0\.7960\.8930\.7910\.904MIMIC\-IIIHeart Failure Mortality0\.7540\.9150\.7490\.955MIMIC\-IIIHospital Readmission0\.8290\.8920\.8290\.875MIMIC\-IIILength\-of\-Stay0\.8400\.8950\.8410\.871MIMIC\-IIIICU Readmission0\.8360\.8920\.8320\.875MIMIC\-IIIHospital Mortality0\.8460\.8730\.8470\.815MIMIC\-IVCOPD Mortality0\.8050\.8850\.8100\.858MIMIC\-IVHeart Failure Mortality0\.8250\.8830\.8220\.866MIMIC\-IVHospital Readmission0\.8450\.9280\.8870\.915MIMIC\-IVLength\-of\-Stay0\.9160\.9020\.9130\.926MIMIC\-IVICU Mortality0\.8560\.8900\.8570\.845MIMIC\-IVICU Readmission0\.7540\.8940\.7490\.912eICUCOPD Mortality0\.8000\.8400\.8090\.865eICUHeart Failure Mortality0\.8320\.8600\.8190\.872eICULength\-of\-Stay0\.8570\.8630\.8650\.863eICUMortality0\.9070\.8840\.9150\.860eICUReadmission0\.8630\.8900\.8540\.885

Table 2:Per\-dataset Overall fidelity for the four generators on tabular \(left\) and time\-series \(right\) tasks\. Bold marks the best model per dataset\.### Statistical Fidelity

Table[2](https://arxiv.org/html/2608.12805#Sx4.T2)reports per\-dataset Overall fidelity for all four generators on both modalities\. Averaging across datasets, CoMed\-TVAE is the most faithful generator on both modalities, attaining the highest mean Overall score \(0\.902 tabular, 0\.887 time\-series\) and winning on the majority of individual datasets\. On tabular data the other three generators are closely grouped \(mean Overall 0\.85–0\.87\), with CoMed\-GaussianCopula \(0\.867\) slightly ahead of the GAN\-based CoMed\-CTGAN \(0\.860\) and CoMed\-CopulaGAN \(0\.848\)\. On time\-series data CoMed\-GaussianCopula is the clear runner\-up \(mean 0\.880\), while CoMed\-CTGAN \(0\.833\) and CoMed\-CopulaGAN \(0\.831\) trail\. Table[1](https://arxiv.org/html/2608.12805#Sx4.T1)isolates the effect of the clinical\-validity layer: it improves Overall fidelity for the GAN\-based CoMed\-CTGAN and CoMed\-CopulaGAN on nearly every affected task \(up to\+0\.098\+0\.098on cervical cancer\), while the CoMed\-TVAE and CoMed\-GaussianCopula models change only marginally\.

### Downstream Utility

Table[3](https://arxiv.org/html/2608.12805#Sx4.T3)reports per\-dataset downstream utility \(AUROC and AUPRC\) for all four synthesizers\. On tabular tasks the reference generator CoMed\-CTGAN preserves most of the real\-data signal \(mean AUC\-ROC utility90\.6%90\.6\\%, median89\.3%89\.3\\%\), and the strongest generator CoMed\-TVAE reaches97\.3%97\.3\\%; several datasets meet or exceed the real baseline \(GBSG100\.5%100\.5\\%, Mammographic Mass100\.0%100\.0\\%\)\. Time\-series tasks are harder and more generator\-sensitive: CoMed\-CTGAN retains a mean AUC\-ROC utility of81\.6%81\.6\\%, whereas CoMed\-TVAE still retains∼95%\{\\sim\}95\\%\. The gap widens sharply under AUC\-PRC, which is sensitive to minority\-class recovery: for CoMed\-CTGAN, tabular utility averages86\.4%86\.4\\%but time\-series falls to64\.0%64\.0\\%, with rare\-outcome ICU mortality tasks retaining only∼30\{\\sim\}30\-45%45\\%\. Thus synthetic data transfers well for static tabular prediction, but the weaker generators lose substantial minority\-class signal on temporal, highly imbalanced ICU tasks\.

Table 3:Per\-dataset downstream utility\. Each cell reportsAUROC±\\pmstd, AUPRC±\\pmstd\(mean±\\pmstd over the five classifiers; Real=\{=\}TRTR, generators=\{=\}TSTR\)\. Best synthetic generator per row \(by AUROC\) inbold\. Binary tasks use standard AUROC/AUPRC; theMulticlassblock uses macro one\-vs\-rest AUROC and macro AUPRC\.![Refer to caption](https://arxiv.org/html/2608.12805v1/figures/J_overall_vs_utility_tvae.png)Figure 5:Overall fidelity vs\. downstream utility \(CoMed\-TVAE AUC\-ROC utility\) across all3737dataset\-task pairs\. Overall fidelity is the strongest single predictor of utility \(Pearsonr=0\.40r=0\.40, Spearmanρ=0\.48\\rho=0\.48,p=0\.014p=0\.014\); Correlation Stability is intermediate \(r=0\.33r=0\.33\) and marginal Distribution Stability weakest \(r=0\.25r=0\.25\), while dataset size is not predictive \(r=−0\.27r=\-0\.27,p=0\.13p=0\.13\)\.The distributions in tabular utility are higher and tighter \(median near93%93\\%across classifier\-dataset pairs\), while time\-series utility is lower and more dispersed\. The choice of downstream classifier matters relatively little, so the data modality, not the classifier, drives utility\. Notably, on the imbalanced ICU tasks a synthetic dataset can retain acceptable AUROC while its AUPRC collapses, so AUROC alone overstates usefulness for rare\-outcome prediction\.

### Does Fidelity Predict Utility?

Finally we relate the two axes across all3737dataset\-task pairs: higher fidelity is associated with higher utility, but the strength depends on the property\. For the reference generator CoMed\-CTGAN, Overall fidelity is the best single predictor \(r=0\.67r=0\.67,ρ=0\.74\\rho=0\.74\), Correlation Stability is intermediate \(r=0\.58r=0\.58\), and marginal Distribution Stability is weakest \(r=0\.45r=0\.45\)\. Dataset size shows no significant relationship \(r=−0\.20r=\-0\.20,p=0\.26p=0\.26\)\. The same ranking of predictors holds for the strongest generator, CoMed\-TVAE \(Figure[5](https://arxiv.org/html/2608.12805#Sx4.F5)\), although the correlation is attenuated \(Overallr=0\.40r=0\.40,ρ=0\.48\\rho=0\.48\) because its near\-ceiling utility leaves little variance for fidelity to explain\. These correlations are positive but imperfect, reinforcing the benchmark’s premise: fidelity is a useful but incomplete proxy, and downstream utility must be measured directly\.

## Conclusion

We presented CoMedBench, a reproducible benchmark that evaluates synthetic clinical data under a common clinical\-validity framework along two axes\-statistical fidelity and downstream task utility\-using a single configuration\-driven engine over three intensive\-care databases and a suite of public clinical and tabular datasets\. Across four generators, no single one dominates: CoMed\-TVAE is the most faithful overall, while the Gaussian\-copula model CoMed\-GC is competitive or best on several small or highly imbalanced cohorts\.

Our central finding concerns usefulness rather than appearance\. Synthetic tabular data preserves most downstream signal \(mean AUC\-ROC utility90\.6%90\.6\\%for CoMed\-CTGAN, up to97\.3%97\.3\\%for CoMed\-TVAE\), and several datasets match the real\-data upper bound\. Temporal ICU tasks are harder and more generator\-sensitive: under CoMed\-CTGAN utility falls to81\.6%81\.6\\%under AUC\-ROC and to64\.0%64\.0\\%under the imbalance\-sensitive AUC\-PRC, though CoMed\-TVAE still retains∼95%\{\\sim\}95\\%under AUC\-ROC\. Relating the two axes, fidelity\-especially the preservation of pairwise correlation structure\-is positively but imperfectly correlated with utility \(for CoMed\-CTGAN, Overallr=0\.67r=0\.67,ρ=0\.74\\rho=0\.74\), while dataset size is not\. Statistical fidelity is therefore a useful but insufficient proxy, and downstream utility must be evaluated directly\.

The practical message is that synthetic data is already a credible substitute for static tabular prototyping and benchmarking, while rare\-event imbalance remains the hardest case\. Because our pipeline summarizes each ICU stay into a per\-stay feature vector, claims about fine\-grained temporal dynamics are out of scope; this motivates comparison against sequence\-native generators\. Integrating privacy tests would complete the fidelity\-utility\-privacy picture needed for clinical deployment\.

## References

- Adisa \(2026\)I\. T\. AdisaAn integrated framework for explainable, fair, and observable hospital readmission prediction: development and validation on mimic\-iv\.External Links:2604\.22535,[Link](https://arxiv.org/abs/2604.22535)Cited by:[Synthetic Data in Downstream Tasks](https://arxiv.org/html/2608.12805#Sx2.SSx2.p2.1)\.
- Buiet al\.\(2024\)H\. Bui, H\. Warrier, and Y\. GuptaBenchmarking with mimic\-iv, an irregular, spare clinical time series dataset\.arXiv preprint arXiv:2401\.15290\.Cited by:[Synthetic Data in Downstream Tasks](https://arxiv.org/html/2608.12805#Sx2.SSx2.p2.1)\.
- Cheet al\.\(2018\)Z\. Che, S\. Purushotham, K\. Cho, D\. Sontag, and Y\. LiuRecurrent neural networks for multivariate time series with missing values\.Scientific Reports8,pp\. 6085\.Cited by:[Synthetic Data in Tabular and Time\-Series Data](https://arxiv.org/html/2608.12805#Sx2.SSx1.p3.1),[Synthetic Data in Downstream Tasks](https://arxiv.org/html/2608.12805#Sx2.SSx2.p2.1)\.
- Chenet al\.\(2021\)R\. J\. Chen, M\. Y\. Lu, T\. Y\. Chen, D\. F\. K\. Williamson, and F\. MahmoodSynthetic data in machine learning for medicine and healthcare\.Nature Biomedical Engineering5\(6\),pp\. 493–497\.External Links:[Document](https://dx.doi.org/10.1038/s41551-021-00751-8),[Link](https://doi.org/10.1038/s41551-021-00751-8),ISSN 2157\-846XCited by:[Synthetic Data in Healthcare AI](https://arxiv.org/html/2608.12805#Sx2.SSx3.p2.1)\.
- Chicco and Jurman \(2020\)D\. Chicco and G\. JurmanMachine learning can predict survival of patients with heart failure from serum creatinine and ejection fraction alone\.BMC medical informatics and decision making20\(1\),pp\. 16\.Cited by:[Datasets, Tasks, and Representation](https://arxiv.org/html/2608.12805#Sx3.SSx2.p1.1)\.
- Guptaet al\.\(2022a\)M\. Gupta, B\. Gallamoza, N\. Cutrona, P\. Dhakal, R\. Poulain, and R\. BeheshtiMIMIC\-iv data pipeline for clinical prediction tasks\.External Links:2204\.13841Cited by:[Synthetic Data in Downstream Tasks](https://arxiv.org/html/2608.12805#Sx2.SSx2.p2.1)\.
- Guptaet al\.\(2022b\)M\. Gupta, B\. Gallamoza, N\. Cutrona, P\. Dhakal, R\. Poulain, and R\. BeheshtiAn extensive data processing pipeline for mimic\-iv\.InMachine learning for health,pp\. 311–325\.Cited by:[Synthetic Data in Downstream Tasks](https://arxiv.org/html/2608.12805#Sx2.SSx2.p2.1)\.
- Habehh and Gohel \(2021\)H\. Habehh and S\. GohelMachine learning in healthcare\.Current genomics22\(4\),pp\. 291–300\.Cited by:[Introduction](https://arxiv.org/html/2608.12805#Sx1.p1.1)\.
- Harutyunyanet al\.\(2019\)H\. Harutyunyan, H\. Khachatrian, D\. C\. Kale, G\. Ver Steeg, and A\. GalstyanMultitask learning and benchmarking with clinical time series data\.Scientific Data6,pp\. 96\.Cited by:[Synthetic Data in Tabular and Time\-Series Data](https://arxiv.org/html/2608.12805#Sx2.SSx1.p3.1),[Synthetic Data in Downstream Tasks](https://arxiv.org/html/2608.12805#Sx2.SSx2.p2.1)\.
- Johnsonet al\.\(2023\)A\. E\. W\. Johnson, L\. Bulgarelli, L\. Shen, A\. Gayles, A\. Shammout, S\. Horng, T\. J\. Pollard, S\. Hao, B\. Moody, B\. Gow, L\. H\. Lehman, L\. A\. Celi, and R\. G\. MarkMIMIC\-iv, a freely accessible electronic health record dataset\.Scientific Data10,pp\. 1\.Cited by:[Introduction](https://arxiv.org/html/2608.12805#Sx1.p1.1),[Synthetic Data in Healthcare AI](https://arxiv.org/html/2608.12805#Sx2.SSx3.p1.1),[Datasets, Tasks, and Representation](https://arxiv.org/html/2608.12805#Sx3.SSx2.p1.1)\.
- Johnsonet al\.\(2016\)A\. E\. W\. Johnson, T\. J\. Pollard, L\. Shen, L\. H\. Lehman, M\. Feng, M\. Ghassemi, B\. Moody, P\. Szolovits, L\. A\. Celi, and R\. G\. MarkMIMIC\-iii, a freely accessible critical care database\.Scientific Data3,pp\. 160035\.Cited by:[Introduction](https://arxiv.org/html/2608.12805#Sx1.p1.1),[Synthetic Data in Healthcare AI](https://arxiv.org/html/2608.12805#Sx2.SSx3.p1.1),[Datasets, Tasks, and Representation](https://arxiv.org/html/2608.12805#Sx3.SSx2.p1.1)\.
- Kakadiaris \(2023\)A\. KakadiarisEvaluating the fairness of the mimic\-iv dataset and a baseline algorithm: application to the icu length of stay prediction\.External Links:2401\.00902,[Link](https://arxiv.org/abs/2401.00902)Cited by:[Synthetic Data in Downstream Tasks](https://arxiv.org/html/2608.12805#Sx2.SSx2.p2.1)\.
- Katzmanet al\.\(2018\)J\. L\. Katzman, U\. Shaham, A\. Cloninger, J\. Bates, T\. Jiang, and Y\. KlugerDeepSurv: personalized treatment recommender system using a cox proportional hazards deep neural network\.BMC medical research methodology18\(1\),pp\. 24\.Cited by:[Datasets, Tasks, and Representation](https://arxiv.org/html/2608.12805#Sx3.SSx2.p1.1)\.
- Leeet al\.\(2022\)J\. Lee, M\. Kim, Y\. Jeong, and Y\. RoDifferentially private normalizing flows for synthetic tabular data generation\.InProceedings of the AAAI Conference on Artificial Intelligence,Cited by:[Synthetic Data in Tabular and Time\-Series Data](https://arxiv.org/html/2608.12805#Sx2.SSx1.p1.1)\.
- Liet al\.\(2021\)F\. Li, H\. Xin, J\. Zhang, M\. Fu, J\. Zhou, and Z\. LianPrediction model of in\-hospital mortality in intensive care unit patients with heart failure: machine learning\-based, retrospective analysis of the mimic\-iii database\.BMJ open11\(7\),pp\. e044779\.Cited by:[Synthetic Data in Downstream Tasks](https://arxiv.org/html/2608.12805#Sx2.SSx2.p2.1)\.
- Linet al\.\(2019\)Y\. Lin, Y\. Zhou, F\. Faghri, M\. J\. Shaw, and R\. H\. CampbellAnalysis and prediction of unplanned intensive care unit readmission using recurrent neural networks with long short\-term memory\.PloS one14\(7\),pp\. e0218942\.Cited by:[Synthetic Data in Downstream Tasks](https://arxiv.org/html/2608.12805#Sx2.SSx2.p2.1)\.
- Patkiet al\.\(2016\)N\. Patki, R\. Wedge, and K\. VeeramachaneniThe synthetic data vault\.InIEEE International Conference on Data Science and Advanced Analytics,pp\. 399–410\.Cited by:[Synthetic Data in Tabular and Time\-Series Data](https://arxiv.org/html/2608.12805#Sx2.SSx1.p1.1)\.
- Patrícioet al\.\(2018\)M\. Patrício, J\. Pereira, J\. Crisóstomo, P\. Matafome, M\. Gomes, R\. Seiça, and F\. CarameloUsing resistin, glucose, age and bmi to predict the presence of breast cancer\.BMC cancer18\(1\),pp\. 29\.Cited by:[Datasets, Tasks, and Representation](https://arxiv.org/html/2608.12805#Sx3.SSx2.p1.1)\.
- Penget al\.\(2022\)S\. Peng, J\. Huang, X\. Liu, J\. Deng, C\. Sun, J\. Tang, H\. Chen, W\. Cao, W\. Wang, X\. Duan,et al\.Interpretable machine learning for 28\-day all\-cause in\-hospital mortality prediction in critically ill patients with heart failure combined with hypertension: a retrospective cohort study based on medical information mart for intensive care database\-iv and eicu databases\.Frontiers in cardiovascular medicine9,pp\. 994359\.Cited by:[Synthetic Data in Downstream Tasks](https://arxiv.org/html/2608.12805#Sx2.SSx2.p2.1)\.
- Piaset al\.\(2025\)T\. S\. Pias, S\. Afrose, M\. D\. Tuli, I\. H\. Trisha, X\. Deng, C\. B\. Nemeroff, and D\. D\. YaoLow responsiveness of machine learning models to critical or deteriorating health conditions\.Communications Medicine5\(1\),pp\. 62\.External Links:[Document](https://dx.doi.org/10.1038/s43856-025-00775-0),[Link](https://doi.org/10.1038/s43856-025-00775-0)Cited by:[Introduction](https://arxiv.org/html/2608.12805#Sx1.p3.1)\.
- Piaset al\.\(2026\)T\. S\. Pias, Y\. Su, X\. Tang, H\. Wang, S\. Faghani, and D\. YaoEnhancing fairness and accuracy in diagnosing type 2 diabetes in young adult population\.IEEE Journal of Biomedical and Health Informatics30\(4\),pp\. 3321–3330\.External Links:[Document](https://dx.doi.org/10.1109/JBHI.2025.3616312)Cited by:[Datasets, Tasks, and Representation](https://arxiv.org/html/2608.12805#Sx3.SSx2.p1.1)\.
- Pollardet al\.\(2018\)T\. J\. Pollard, A\. E\. W\. Johnson, J\. D\. Raffa, L\. A\. Celi, R\. G\. Mark, and O\. BadawiThe eicu collaborative research database, a freely available multi\-center database for critical care research\.Scientific Data5,pp\. 180178\.Cited by:[Introduction](https://arxiv.org/html/2608.12805#Sx1.p1.1),[Synthetic Data in Healthcare AI](https://arxiv.org/html/2608.12805#Sx2.SSx3.p1.1),[Datasets, Tasks, and Representation](https://arxiv.org/html/2608.12805#Sx3.SSx2.p1.1)\.
- Qiuet al\.\(2022\)W\. Qiu, H\. Chen, A\. B\. Dincer, S\. Lundberg, M\. Kaeberlein, and S\. LeeInterpretable machine learning prediction of all\-cause mortality\.Communications medicine2\(1\),pp\. 125\.Cited by:[Synthetic Data in Downstream Tasks](https://arxiv.org/html/2608.12805#Sx2.SSx2.p2.1)\.
- Ramanaet al\.\(2011\)B\. V\. Ramana, M\. S\. P\. Babu, N\. Venkateswarlu,et al\.A critical study of selected classification algorithms for liver disease diagnosis\.International Journal of Database Management Systems3\(2\),pp\. 101–114\.Cited by:[Datasets, Tasks, and Representation](https://arxiv.org/html/2608.12805#Sx3.SSx2.p1.1)\.
- Stracket al\.\(2014\)B\. Strack, J\. P\. DeShazo, C\. Gennings, J\. L\. Olmo, S\. Ventura, K\. J\. Cios, and J\. N\. CloreImpact of hba1c measurement on hospital readmission rates: analysis of 70,000 clinical database patient records\.BioMed research international2014\(1\),pp\. 781670\.Cited by:[Datasets, Tasks, and Representation](https://arxiv.org/html/2608.12805#Sx3.SSx2.p1.1)\.
- Tanget al\.\(2020\)S\. Tang, P\. Davarmanesh, Y\. Song, D\. Koutra, M\. W\. Sjoding, and J\. WiensDemocratizing ehr analyses with fiddle: a flexible data\-driven preprocessing pipeline for structured clinical data\.Journal of the American Medical Informatics Association27\(12\),pp\. 1921–1934\.Cited by:[Preprocessing and Metadata](https://arxiv.org/html/2608.12805#Sx3.SSx3.p1.1)\.
- Tuckeret al\.\(2020\)A\. Tucker, Z\. Wang, Y\. Rotalinti, and P\. MylesGenerating high\-fidelity synthetic patient data for assessing machine learning healthcare software\.npj Digital Medicine3\(1\),pp\. 147\.External Links:[Document](https://dx.doi.org/10.1038/s41746-020-00353-9),[Link](https://doi.org/10.1038/s41746-020-00353-9),ISSN 2398\-6352Cited by:[Synthetic Data in Healthcare AI](https://arxiv.org/html/2608.12805#Sx2.SSx3.p2.1)\.
- Xieet al\.\(2019\)Z\. Xie, O\. Nikolayeva, J\. Luo, and D\. LiBuilding risk prediction models for type 2 diabetes using machine learning techniques\.Preventing chronic disease16,pp\. E130\.Cited by:[Datasets, Tasks, and Representation](https://arxiv.org/html/2608.12805#Sx3.SSx2.p1.1)\.
- Xuet al\.\(2019\)L\. Xu, M\. Skoularidou, A\. Cuesta\-Infante, and K\. VeeramachaneniModeling tabular data using conditional gan\.InAdvances in Neural Information Processing Systems,Cited by:[Synthetic Data in Tabular and Time\-Series Data](https://arxiv.org/html/2608.12805#Sx2.SSx1.p1.1),[Synthesis: Generator Families](https://arxiv.org/html/2608.12805#Sx3.SSx5.p1.1)\.