Post-Generation Curation of Synthetic Images via Homogeneous-Heterogeneous Splitting

# Post-Generation Curation of Synthetic Images via Homogeneous-Heterogeneous Splitting
Source: [https://arxiv.org/html/2607.02637](https://arxiv.org/html/2607.02637)
Disheng Liu Tuo Liang Chaoda Song Yu Yin Department of Computer and Data Sciences Case Western Reserve University Cleveland, OH, USA \{dxl952,txl859,cxs965,yxy1421\}@case\.edu

###### Abstract

Recent generative models can produce high\-quality synthetic images, offering scalable training training data for data\-hungry models\. Existing approaches to exploiting this potential typically involve 1\) training or fine\-tuning generators, or 2\) using lightweight post\-hoc adaptation like prompt engineering or inference\-time guidance, making them generator\-specific and expertise\-intensive\. We study a complementary question:*given a fixed pool of generated images, can downstream utility be improved purely by selecting an informative subset?*The answer is yes\. We show that effective selection must counter a structural bias of modern generators: they tend to over\-produce canonical modes of each class while under\-representing intra\-class variation\. Building on this insight, we split each real class into a*canonical Homogeneous*\(\\ho\)*subset*and a*non\-redundant Heterogeneous*\(\\he\)*subset*, then score synthetic images by a fidelity\-diversity criterion that rewards semantic alignment while penalizing canonical redundancy\. The method is generator\-agnostic and requires no retraining\. Across multiple benchmarks, it consistently outperforms state\-of\-the\-art data selection baselines and matches the real\-data performance with up to40%40\\%fewer synthetic samples\. The same criterion remains effective when applied on top of stronger task\-tuned generators, with gains on both classification and segmentation tasks\. Post\-generation selection is therefore not a substitute for better generators, but a complementary mechanism for improving the utility of synthetic data\.

## 1Introduction

Generative models \(GMs\) produce high\-quality synthetic data and offer a promising solution for data scarcity in data\-intensive AI\. Recent studies demonstrate that GMs can effectively replicate datasets \(\\egCIFAR\-10, ImageNet\) and improve downstream models trained on such synthetic data in generalization, transferability, and in\-domain accuracy\[[74](https://arxiv.org/html/2607.02637#bib.bib62),[49](https://arxiv.org/html/2607.02637#bib.bib35),[4](https://arxiv.org/html/2607.02637#bib.bib38),[23](https://arxiv.org/html/2607.02637#bib.bib61),[38](https://arxiv.org/html/2607.02637#bib.bib66),[17](https://arxiv.org/html/2607.02637#bib.bib69)\]\.

However, practical challenges remain: 1\) synthetic datasets may contain low\-fidelity or mislabeled samples, introducing harmful noise; 2\) Inadvertently over\-representing dominant patterns could transfer inherent biases to downstream models; 3\) While scaling the data alleviates these issues, it incurs higher computational overhead and longer training times\.

To harness synthetic data, researchers are prioritizing improved generation techniques that yield more faithful and diverse data\. These approaches can be broadly categorized into: 1\) training\- or fine\-tuning\-based methods\[[4](https://arxiv.org/html/2607.02637#bib.bib38),[31](https://arxiv.org/html/2607.02637#bib.bib4)\], which adapt generators to better match target distributions; 2\) lightweight strategies like guidance\[[69](https://arxiv.org/html/2607.02637#bib.bib104),[33](https://arxiv.org/html/2607.02637#bib.bib102),[39](https://arxiv.org/html/2607.02637#bib.bib3),[14](https://arxiv.org/html/2607.02637#bib.bib29),[34](https://arxiv.org/html/2607.02637#bib.bib28)\]or prompting\[[50](https://arxiv.org/html/2607.02637#bib.bib80),[46](https://arxiv.org/html/2607.02637#bib.bib106),[15](https://arxiv.org/html/2607.02637#bib.bib82)\]\. While effective, these approaches face distinct trade\-offs: training\-based methods incur substantial computational overhead due to the optimization of large\-scale models, whereas lightweight strategies often rely on model\-specific configurations, significant domain expertise, or extensive empirical tuning\. Alternatively, data selection\[[18](https://arxiv.org/html/2607.02637#bib.bib109),[2](https://arxiv.org/html/2607.02637#bib.bib107)\]separates quality control from the image generation, bypassing the typical pitfalls of steering\-based methods\. This data\-centric focus serves as a practical and complementary refinement stage, ensuring high\-quality synthetic outputs even after specialized, task\-guided generation phases\.

![Refer to caption](https://arxiv.org/html/2607.02637v1/x1.png)Figure 1:Data selection while considering both fidelity and diversity\. Real data \(targeted distribution\)is divided into the\\ho\-Set and the\\he\-Set\. Synthetic instances are then scored by referring to these two partitions for subsequent selection\.Aligning curated data with the target distribution is a promising post\-processing principle\. Image–Label Alignment methods\[[23](https://arxiv.org/html/2607.02637#bib.bib61),[11](https://arxiv.org/html/2607.02637#bib.bib63),[73](https://arxiv.org/html/2607.02637#bib.bib33)\]assume that high\-quality data align strictly with the generative labels, employing pre\-trained discriminators to filter out noisy samples; Image–Image Alignment\[[36](https://arxiv.org/html/2607.02637#bib.bib14),[40](https://arxiv.org/html/2607.02637#bib.bib56)\]prioritizes synthetic samples that closely resemble real images\. Both strategies emphasize fidelity, but largely neglect diversity, reducing the utility of synthetic data\[[17](https://arxiv.org/html/2607.02637#bib.bib69)\]due to repeated patterns and missing novel information\.

Motivated by these limitations, we propose afidelity–diversitybalanced post\-generation curation framework to enhance the utility of synthetic imagery for downstream tasks\. In the curation pipeline, we first split the real dataset into two subsets: aHomogeneousset characterized by high internal similarity, and aHeterogeneousset enriched with variation\. Based on this partitioning, we design a scoring mechanism to identify and select desired synthetic instances, balancing high semantic quality with representational diversity, shown as Fig\.[1](https://arxiv.org/html/2607.02637#S1.F1)\)\. Concretely, we compute two complementary metrics: 1\)fidelity scores, measuring semantic similarity to real samples; 2\)diversity scores, quantifying the deviation from repetitive patterns\. Combining such metrics, we curated the final desired synthetic pool\.

To quantify the utility of our curation strategy, experiments are implemented across diverse datasets \(e\.g\., CIFAR\-10, ImageNet, etc\.\) and architectures \(e\.g\., ResNet, ViT\)\. We synthesize data pools using generators trained on target distributions, then apply the curation method to train downstream models for in\- and out\-of\-distribution testings\. Furthermore, we apply the curation as a post\-process after optimized generations for downstream tasks\. Empirical results demonstrate the validity of our post\-generation processing under the different settings\. In summary, our contributions are as follows:

1. 1\.A nearest\-neighbor\-cover partition \(\\hohe\) of real data with a minimality guarantee \(Prop\.[1](https://arxiv.org/html/2607.02637#Thmproposition1)\), giving an explicit notion of canonical vs\. non\-redundant samples\.
2. 2\.A principled, post\-generation selection strategy that jointly quantifies and balances fidelity and diversity\. The framework is generator\-agnostic, requiring only a synthetic data pool, and avoids costly retraining or fine\-tuning of generators\.
3. 3\.Extensive experiments validate the approach across classification and segmentation benchmarks with multiple generators and backbones\. Our method consistently improves in\-domain accuracy and OOD robustness over baselines, plus an additional plug\-in evaluation on top of generator\-side intervention methods\.

## 2Related Works

### 2\.1Curating Synthetic Data for downstream utility

Synthetic data has emerged as a promising solution to data scarcity in the AI era\[[54](https://arxiv.org/html/2607.02637#bib.bib84),[43](https://arxiv.org/html/2607.02637#bib.bib83),[64](https://arxiv.org/html/2607.02637#bib.bib23)\]\. Recent studies show that models trained on synthetic data can learn robust visual representations\[[23](https://arxiv.org/html/2607.02637#bib.bib61),[75](https://arxiv.org/html/2607.02637#bib.bib51),[57](https://arxiv.org/html/2607.02637#bib.bib52),[20](https://arxiv.org/html/2607.02637#bib.bib54),[56](https://arxiv.org/html/2607.02637#bib.bib53)\]\. Moreover, augmenting real datasets with synthetic samples has been shown to further improve model performance\[[49](https://arxiv.org/html/2607.02637#bib.bib35),[70](https://arxiv.org/html/2607.02637#bib.bib50),[44](https://arxiv.org/html/2607.02637#bib.bib49)\]\. However, the distribution gap between real and synthetic data\[[17](https://arxiv.org/html/2607.02637#bib.bib69),[21](https://arxiv.org/html/2607.02637#bib.bib77)\]highlights the need for careful tailoring\[[38](https://arxiv.org/html/2607.02637#bib.bib66),[64](https://arxiv.org/html/2607.02637#bib.bib23),[63](https://arxiv.org/html/2607.02637#bib.bib74),[19](https://arxiv.org/html/2607.02637#bib.bib24)\]to fully exploit the potential of synthetic data\.

Intervening before or during generation is an effective way to improve the utility of synthetic data\. Fine\-tuning\-based methods adapt generators to target distributions or downstream tasks\[[4](https://arxiv.org/html/2607.02637#bib.bib38),[31](https://arxiv.org/html/2607.02637#bib.bib4)\], while prompt\-, guidance\-, and sampling\-based methods steer generated samples toward desired classes, attributes, or harder variations\[[69](https://arxiv.org/html/2607.02637#bib.bib104),[33](https://arxiv.org/html/2607.02637#bib.bib102),[39](https://arxiv.org/html/2607.02637#bib.bib3),[46](https://arxiv.org/html/2607.02637#bib.bib106)\]\. Although these approaches can improve the utility of synthetic data, they often require generator access, task\-specific tuning, prompt expertise, or additional generation cost\. In contrast, post\-generation curation offers a practical and scalable alternative to generation\-time intervention for improving synthetic data utility\.

### 2\.2Post\-Generation Synthetic Data Curations

Unlike real data pruning\[[2](https://arxiv.org/html/2607.02637#bib.bib107),[1](https://arxiv.org/html/2607.02637#bib.bib30),[7](https://arxiv.org/html/2607.02637#bib.bib31)\], synthetic data usage involves generator bias, distribution drift, and mode collapse\. Such artifacts require careful curation to steer synthetic pool toward the target\.

Fidelity\-Guided Curation\.High\-fidelity generations ensure semantic correctness, making fidelity\-based selection effective\. Such curation strategies can be broadly categorized into two main approaches:1\) Image\-Label Alignment:pretrained or task\-specific models are used to filter low\-quality samples by discarding those misclassified within top\-kkpredictions\[[5](https://arxiv.org/html/2607.02637#bib.bib34),[72](https://arxiv.org/html/2607.02637#bib.bib16),[10](https://arxiv.org/html/2607.02637#bib.bib17),[60](https://arxiv.org/html/2607.02637#bib.bib18)\]\. Concretely, CLIP\[[45](https://arxiv.org/html/2607.02637#bib.bib32)\]is used to assess image\-label semantic alignment\[[23](https://arxiv.org/html/2607.02637#bib.bib61),[40](https://arxiv.org/html/2607.02637#bib.bib56),[55](https://arxiv.org/html/2607.02637#bib.bib101)\];2\) Image\-Image Alignment:this approach quantifies similarity between synthetic and real images to filter out low\-quality data that deviate from the real distribution\[[36](https://arxiv.org/html/2607.02637#bib.bib14),[40](https://arxiv.org/html/2607.02637#bib.bib56),[41](https://arxiv.org/html/2607.02637#bib.bib59),[8](https://arxiv.org/html/2607.02637#bib.bib60)\]\. Specificly, clustering\-based curation\[[40](https://arxiv.org/html/2607.02637#bib.bib56)\]use real\-data cluster centroids as anchors to retrieve synthetic samples\. While straightforward, these methods rely heavily on visual similarity and risk reducing data diversity for downstream tasks\.

Diversity\-Guided Curations\.As generative images become increasingly realistic, diversity is key for downstream usage\. Prior works enhance diversity by varying prompts\[[50](https://arxiv.org/html/2607.02637#bib.bib80)\], applying text\-conditioned augmentation\[[15](https://arxiv.org/html/2607.02637#bib.bib82),[13](https://arxiv.org/html/2607.02637#bib.bib110)\], using textual inversion\[[59](https://arxiv.org/html/2607.02637#bib.bib81)\], or conditioning on classifier outputs\[[24](https://arxiv.org/html/2607.02637#bib.bib100)\]\. While effective, these methods require fine\-tuning or prompt engineering and focus on generation\-time diversity, offering little guidance on how to efficiently leverage existing synthetic datasets\. Building the priors, we curate synthetic datasets by jointly considering fidelity and diversity, without modifying generative model output, forming a post\-generation data curation pipeline\.

## 3Methods

Overview\.We target the following*post\-generation curation*problem and assume only sample access to the generator\. Given \(i\) a labeled real reference set𝒟R=\{\(Ii,yi\)\}i=1\|𝒟R\|\\mathcal\{D\}\_\{R\}=\\\{\(I\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{\|\\mathcal\{D\}\_\{R\}\|\}, \(ii\) a synthetic pool𝒟S=\{I~j\}j=1\|𝒟S\|\\mathcal\{D\}\_\{S\}=\\\{\\tilde\{I\}\_\{j\}\\\}\_\{j=1\}^\{\|\\mathcal\{D\}\_\{S\}\|\}with class assignments inherited from the generator’s conditioning, and \(iii\) a selection budgetkk\(possibly much larger than\|𝒟R\|\|\\mathcal\{D\}\_\{R\}\|\), the goal is to select𝒜⊆𝒟S\\mathcal\{A\}\\subseteq\\mathcal\{D\}\_\{S\}so that the selected synthetic training set improves downstream classification on the real distribution\.

Our method has two steps\. First, we split the real reference images into a*Homogeneous*subsetℐH​O\\mathcal\{I\}\_\{HO\}of local representatives and a*Heterogeneous*subsetℐH​E\\mathcal\{I\}\_\{HE\}of non\-redundant variation \(Sec\.[3\.1](https://arxiv.org/html/2607.02637#S3.SS1)\)\. Second, we allocate the synthetic budget across these two subsets and score each synthetic candidate with a partition\-conditioned fidelity–diversity criterion \(Sec\.[3\.2](https://arxiv.org/html/2607.02637#S3.SS2)\)\. The key design principle is that the selected set should match both the canonical component and the heterogeneous component of the real class, instead of ranking all synthetic images against a single pooled reference distribution\.

### 3\.1Categorizing Homogeneous \(\\ho\) and Heterogeneous \(\\he\) Samples

To guide the selection of synthetic data, we first categorize target real distribution into two distinct sets:Homogeneous \(\\ho\)andHeterogeneous \(\\he\)\.\\hoinstancesrepresent the canonical semantics, exhibiting high intra\-class similarity in feature space\.\\heinstancescapture greater variation, including less typical instances that contribute to diversity\.

Identifying\\hoand\\heInstances\.We construct the\\ho/\\hesplit from the directed11\-nearest\-neighbor \(1\-NN\) graph of each real class in feature space\. The construction is local, non\-parametric, and costsO​\(n2​d\)O\(n^\{2\}d\)for a class withnnexamples andddis the feature dimension of input\. Unlike a centroid\-near/far split \(\\egkk\-means, core\-set partitions\), it does not impose a global axis through the class:ℐℋ​𝒪\\mathcal\{I\_\{HO\}\}contains examples that other examples use as their closest local representatives, whileℐℋ​ℰ\\mathcal\{I\_\{HE\}\}contains examples with zero in\-degree in this local graph\.

Given imagesℐ=\{I1,…,In\}\\mathcal\{I\}=\\\{I\_\{1\},\\ldots,I\_\{n\}\\\}, we extractedℓ2\\ell\_\{2\}\-normalized featuresℱ=\{f1,…,fn\}\\mathcal\{F\}=\\\{f\_\{1\},\\ldots,f\_\{n\}\\\}using a pretrained encoder \(\\egMoCo v3\[[12](https://arxiv.org/html/2607.02637#bib.bib41)\]\)\. We use cosine distanced​\(fi,fj\)=1−⟨fi,fj⟩d\(f\_\{i\},f\_\{j\}\)=1\-\\langle f\_\{i\},f\_\{j\}\\ranglethroughout\. For eachIiI\_\{i\}, we define its within\-class nearest neighbor

j∗​\(i\)=arg⁡maxj≠i⁡⟨fi,fj⟩\.j^\{\\ast\}\(i\)=\\arg\\max\_\{j\\neq i\}\\langle f\_\{i\},f\_\{j\}\\rangle\.\(1\)Therefore, the\\hoset is the image of this map and the\\heset is its complement:

ℐℋ​𝒪=\{Ii:∃j≠i,j∗​\(j\)=i\},ℐℋ​ℰ=ℐ∖ℐℋ​𝒪\.\\mathcal\{I\_\{HO\}\}=\\\{I\_\{i\}:\\exists j\\neq i,\\;j^\{\\ast\}\(j\)=i\\\},\\qquad\\mathcal\{I\_\{HE\}\}=\\mathcal\{I\}\\setminus\\mathcal\{I\_\{HO\}\}\.\(2\)Equivalently, in the directed 1\-NN graph with edgesIi→Ij∗​\(i\)I\_\{i\}\\to I\_\{j^\{\\ast\}\(i\)\},ℐℋ​𝒪\\mathcal\{I\_\{HO\}\}is the set of nodes with positive in\-degree andℐℋ​ℰ\\mathcal\{I\_\{HE\}\}is the set of zero in\-degree nodes\.

###### Proposition 1\(ℐH​O\\mathcal\{I\}\_\{HO\}as a minimal nearest\-neighbor cover\)\.

Assume every sample has a unique within\-class nearest neighbor\. ThenℐH​O\\mathcal\{I\}\_\{HO\}is the unique inclusion\-minimal subsetA⊆ℐA\\subseteq\\mathcal\{I\}such that

minIi∈A,i≠j⁡d​\(fj,fi\)=minIi∈ℐ,i≠j⁡d​\(fj,fi\),∀Ij∈ℐ\.\\min\_\{I\_\{i\}\\in A,\\,i\\neq j\}d\(f\_\{j\},f\_\{i\}\)\\;=\\;\\min\_\{I\_\{i\}\\in\\mathcal\{I\},\\,i\\neq j\}d\(f\_\{j\},f\_\{i\}\),\\quad\\forall\\,I\_\{j\}\\in\\mathcal\{I\}\.\(3\)Thus, withε=maxj⁡d​\(fj,fj∗​\(j\)\)\\varepsilon=\\max\_\{j\}d\(f\_\{j\},f\_\{j^\{\\ast\}\(j\)\}\), every real sample has anℐH​O\\mathcal\{I\}\_\{HO\}representative within distanceε\\varepsilon\.

Proposition[1](https://arxiv.org/html/2607.02637#Thmproposition1)formalizes the representative role ofℐℋ​𝒪\\mathcal\{I\_\{HO\}\}: replacing the full class byℐℋ​𝒪\\mathcal\{I\_\{HO\}\}preserves the nearest\-neighbor reconstruction cost of every training image\. The complementℐℋ​ℰ\\mathcal\{I\_\{HE\}\}should therefore not be read as noise or outliers\. It is the part of the class that is not needed as a nearest\-neighbor representative for other examples, and is precisely where non\-redundant variation can be lost when a generator over\-produces canonical images\.

![Refer to caption](https://arxiv.org/html/2607.02637v1/x2.png)Figure 2:PCA of the CIFAR\-10 “horse” class under different partitioning rules\. The 1\-NNHO\-HE split\(top\)preserves local neighborhood, whereascentroid\-based split\(bottom\)cuts a class along a single global axis\.Empiricalℐℋ​𝒪−ℐℋ​ℰ\\bm\{\\mathcal\{I\_\{HO\}\}\}\-\\bm\{\\mathcal\{I\_\{HE\}\}\}Analysis and Impact\.As visualized in Fig\.[2](https://arxiv.org/html/2607.02637#S3.F2), the\\hohesplit preserves local neighborhood and keeps each semantic mode mixed across both subsets, avoiding mode imbalance during guided selection\. In contrast, centroid\-based alternatives that define near/far canonical patterns tend to isolate individual mode regions, yielding suboptimal selection references\. Without a complete semantic landscape, prioritizing any single mode during synthetic data curation inevitably overlooks parts of the real distribution, leading to information loss\. We also show\\ho\-\\hepartition instances in CIFAR\-10 \(Fig\.[4](https://arxiv.org/html/2607.02637#S3.F4)\)\. Because\\hosamples are retrieved as local representatives, they have higher intra\-class similarity than\\hesamples, both visually and quantitatively \(more cases in Appendix Fig\.[10](https://arxiv.org/html/2607.02637#A1.F10)and Tab\.[7](https://arxiv.org/html/2607.02637#A1.T7)\)\.

Although bothℐH​O\\mathcal\{I\}\_\{HO\}andℐH​E\\mathcal\{I\}\_\{HE\}encompass all distribution modes, we observe an interesting divergence in downstream behaviors: 1\)GMs preferentially learn and reproduceℐH​O\\bm\{\\mathcal\{I\}\_\{HO\}\}\. As quantified in Fig\.[4](https://arxiv.org/html/2607.02637#S3.F4), synthetic data disproportionately mirrorsℐH​O\\mathcal\{I\}\_\{HO\}overℐH​E\\mathcal\{I\}\_\{HE\}\. This inherent preference indicates that GMs gravitate toward repeated canonical patterns withinℐH​O\\mathcal\{I\}\_\{HO\}, which may explain the persistent diversity limitations observed in modern generative models\. 2\)Discriminators exhibit a similar performance bias, excelling onℐH​O\\bm\{\\mathcal\{I\}\_\{HO\}\}while struggling withℐH​E\\bm\{\\mathcal\{I\}\_\{HE\}\}\. This discrepancy indicates that, despite sharing underlying semantics,ℐH​E\\mathcal\{I\}\_\{HE\}contains inherently more challenging cases that discriminators fail to model effectively \(empirical supports across datasets is provided in the Appendix[12](https://arxiv.org/html/2607.02637#A3.F12), Fig\.[13](https://arxiv.org/html/2607.02637#A3.F13)\)\.

![Refer to caption](https://arxiv.org/html/2607.02637v1/x3.png)Figure 3:\\hoand\\heinstances in CIFAR\-10\.Avg\. Sim\.represents the average similarity between the images in each row and the entire class\.\\hoinstances are more representative, expressing the core semantics\.\\heinstances are more diverse, capturing a broader range of variations \(More examples are in the Appendix Fig\.[10](https://arxiv.org/html/2607.02637#A1.F10)\)\.
![Refer to caption](https://arxiv.org/html/2607.02637v1/x4.png)Figure 4:Similarity between the synthetic and\\ho,\\hesplits\. We measure cosine similarity between\\ho/\\hesubsets and generations of models trained by each real dataset\. Synthetic data consistently more closely resembles\\hothan\\he\.

### 3\.2Synthetic Data Selection Strategy

Given the\\hoand\\hesets, we propose a synthetic data selection strategy to identify desirable instances from a fixed synthetic pool\. The core objective is to ensure high semantic fidelity while enhancing diversity by prioritizing samples that diverge from the dominant patterns in\\ho\.

Partitioned Selection Strategy\.Unlike prior methods that treat the entire real dataset as a single reference for data selection, we propose a partitioned selection approach\. We treat\\hoand\\heas separate partitions and select synthetic instances by referring to real images within each partition under the same principle: desired synthetics should be sufficiently close to their corresponding real instances, while prioritizing those that deviate from canonical representations\. By combining the selection results from\\hoand\\heinstances, we obtain the final selected pool\.

Concretely, we compute a selection scoreSpS^\{p\}for each synthetic sample based on its correlation with partitionp∈\{\\ho,\\he\}p\\in\\\{\\ho,\\he\\\}\. Samples are ranked within each partition, and the instances with highSpS^\{p\}get retrieved\. Alg\.[1](https://arxiv.org/html/2607.02637#alg1)for the pseudo‑code of our selection procedure\.

0:Feat\. maps

ℱ\\ho∈ℝn×d\\mathcal\{F\}^\{\\ho\}\\\!\\in\\\!\\mathbb\{R\}^\{n\\times d\},

ℱ\\he∈ℝm×d\\mathcal\{F\}^\{\\he\}\\\!\\in\\\!\\mathbb\{R\}^\{m\\times d\},

ℱs​y​n∈ℝs×d\\mathcal\{F\}^\{syn\}\\\!\\in\\\!\\mathbb\{R\}^\{s\\times d\}; centroid

𝒞\\ho∈ℝd\\mathcal\{C\}^\{\\ho\}\\\!\\in\\\!\\mathbb\{R\}^\{d\}; weight

α∈\[0,1\]\\alpha\\in\[0,1\]; size

kk
10:Selected indices

ℐselected\\mathcal\{I\}\_\{\\text\{selected\}\}
1:\{Step 1: Construct reference anchors\}

2:

ℛ\\ho←𝒞\\ho\\mathcal\{R\}^\{\\ho\}\\leftarrow\\mathcal\{C\}^\{\\ho\}
3:

idx←argmax⁡\(ℱ\\he​\(ℱ\\ho\)⊤,dim=1\)\\mathrm\{idx\}\\leftarrow\\operatorname\{argmax\}\(\\mathcal\{F\}^\{\\he\}\(\\mathcal\{F\}^\{\\ho\}\)^\{\\top\},\\text\{dim\}=1\)
24:

ℛ\\he←ℱ\\ho​\[idx\]\\mathcal\{R\}^\{\\he\}\\leftarrow\\mathcal\{F\}^\{\\ho\}\[\\mathrm\{idx\}\]\{Align\\hefeatures to\\ho\}

5:\{Step 2: Score calculation and selection\}

6:for

p∈\{\\ho,\\he\}p\\in\\\{\\ho,\\he\\\}do

7:

Sfidp←cos⁡\(ℱs​y​n,ℱp\)S^\{p\}\_\{\\text\{fid\}\}\\leftarrow\\cos\(\\mathcal\{F\}^\{syn\},\\mathcal\{F\}^\{p\}\)\{Fidelity\}

8:

Sdivp←−cos⁡\(ℛp−ℱp,ℱs​y​n−ℱp\)S^\{p\}\_\{\\text\{div\}\}\\leftarrow\-\\cos\(\\mathcal\{R\}^\{p\}\-\\mathcal\{F\}^\{p\},\\;\\mathcal\{F\}^\{syn\}\-\\mathcal\{F\}^\{p\}\)\{Diversity\}

9:

Sp←α​Sdivp\+\(1−α\)​SfidpS^\{p\}\\leftarrow\\alpha S^\{p\}\_\{\\text\{div\}\}\+\(1\-\\alpha\)S^\{p\}\_\{\\text\{fid\}\}
10:

ℐtopp←Topk⁡\(Sp\)\\mathcal\{I\}^\{p\}\_\{\\text\{top\}\}\\leftarrow\\operatorname\{Top\}\_\{k\}\(S^\{p\}\)
11:endfor

12:returnSelected set

𝒜←ℐtop\\ho∪ℐtop\\he\\mathcal\{A\}\\leftarrow\\mathcal\{I\}^\{\\ho\}\_\{\\text\{top\}\}\\cup\\mathcal\{I\}^\{\\he\}\_\{\\text\{top\}\}

Algorithm 1Data Selection with\\hoand\\heScoring Mechanism for Each Partition\.Our selection mechanism balances two competing objectives: thefidelity score, measuring how closely a synthetic instance resembles the real, and thediversity score, measuring how much it deviates from canonical patterns in the real dataset\.

1\) Fidelity Score\.The fidelity scoreSfidpS^\{p\}\_\{\\text\{fid\}\}quantifies how well the synthetic samples align with the real distribution of partitionpp\. We measure this using the cosine similarity between the synthetic featuresℱs​y​n\\mathcal\{F\}^\{syn\}and the real featuresℱp\\mathcal\{F\}^\{p\}, as defined in Eq\.[4](https://arxiv.org/html/2607.02637#S3.E4)\. A higher fidelity score indicates that the synthetic samples closely resemble the real data in the corresponding partition:

Sfidp=cos⁡\(ℱs​y​n,ℱp\)\.S^\{p\}\_\{\\text\{fid\}\}=\\cos\(\\mathcal\{F\}^\{syn\},\\mathcal\{F\}^\{p\}\)\.\(4\)
![Refer to caption](https://arxiv.org/html/2607.02637v1/x5.png)Figure 5:Illustration of the computation ofSfidpS\_\{\\text\{fid\}\}^\{p\}andSdivpS\_\{\\text\{div\}\}^\{p\}in partitionpp\. Fidelity is assessed ascos⁡\(ℱisyn,ℱip\)\\cos\\left\(\\mathcal\{F\}\_\{i\}^\{\\text\{syn\}\},\\mathcal\{F\}^\{p\}\_\{i\}\\right\), while diversity is measured by the angle between vectors ending atℱisyn\\mathcal\{F\}\_\{i\}^\{\\text\{syn\}\}andℛip\\mathcal\{R\}^\{p\}\_\{i\}\. In the diagram, Syn\. instance 1 exhibits greater diversity than Syn\. instance 2\.2\) Diversity Score\.The diversity scoreSdivpS^\{p\}\_\{\\text\{div\}\}is designed to encourage the selected synthetic samples to deviate from canonical patterns\. To achieve this, we define a subset\-specific reference anchorℛp\\mathcal\{R\}^\{p\}\. For the\\hosubset, the anchor is the centroid of the real features \(ℛ\\ho=𝒞\\ho\\mathcal\{R\}^\{\\ho\}=\\mathcal\{C\}^\{\\ho\}\)\. For the\\hesubset, the anchor is the nearest matching feature in the\\hosubset \(ℛ\\he=ℱ\\ho​\[idx\]\\mathcal\{R\}^\{\\he\}=\\mathcal\{F\}^\{\\ho\}\[\\mathrm\{idx\}\]\)\. The diversity metric is then formulated as the negative cosine similarity between two directional vectors, as defined in Eq\.[5](https://arxiv.org/html/2607.02637#S3.E5): one from the real instanceℱp\\mathcal\{F\}^\{p\}to the anchorℛp\\mathcal\{R\}^\{p\}, and the other from the same real instance to the synthetic dataℱs​y​n\\mathcal\{F\}^\{syn\}\(representative visual cases are shown in Fig\.[14](https://arxiv.org/html/2607.02637#A4.F14)of the Appendix\)\.

Sdivp=−cos⁡\(ℛp−ℱp,ℱs​y​n−ℱp\)\.S^\{p\}\_\{\\text\{div\}\}=\-\\cos\(\\mathcal\{R\}^\{p\}\-\\mathcal\{F\}^\{p\},\\;\\mathcal\{F\}^\{syn\}\-\\mathcal\{F\}^\{p\}\)\.\(5\)As illustrated in Fig\.[5](https://arxiv.org/html/2607.02637#S3.F5), if these two vectors point in opposite directions, the cosine similarity is negative, leading to a higher diversity score\. If the vectors are aligned, the synthetic sample is closer to the homogeneous distribution, resulting in a lower diversity score\.

3\) Total Score\.To control the trade\-off between fidelity and diversity, we define the score as:

Sp=α⋅Sdivp\+\(1−α\)⋅Sfidp,\{S\}^\{p\}=\\alpha\\cdot S\_\{\\text\{div\}\}^\{p\}\+\(1\-\\alpha\)\\cdot S^\{p\}\_\{\\text\{fid\}\},\(6\)whereα\\alphais a hyperparameter that determines the selection priority\.α=0\\alpha=0\(MaxSim\) prioritizes fidelity, ensuring that synthetic instances closely resemble real ones\.α=1\\alpha=1\(MaxDiv\) prioritizes diversity, encouraging synthetic samples that deviate from the dominant patterns of real data\. Adjustingα\\alphaenables flexible control over the selection behavior to match downstream needs\.

Practical Considerations\.\(a\) Complexity\.Building the 1\-NN graph costsO​\(n2​d\)O\(n^\{2\}d\)after feature extraction, wherennis number of real samples,ddis feature dimension\. Scoring is implemented as batched matrix multiplication and costsO​\(\|𝒟S\|​n​d\)O\(\|\\mathcal\{D\}\_\{S\}\|nd\), where\|𝒟S\|\|\\mathcal\{D\}\_\{S\}\|denotes the number of synthetic candidates\.\(b\) Encoder choice\.The split is relative to encoderϕ\\phi, so a domain\-mismatched encoder can degrade curation\. Sec\.[5](https://arxiv.org/html/2607.02637#S5)compares representative encoders and shows that the gains are not tied to a single feature extractor\.\(c\) No balanced\\hoherequirement\.The\\hohesplit is induced by the 1\-NN graph; it does not assume a target ratio or require pre\-labeled canonical examples\. Unequal partition sizes only change how the synthetic budget is allocated across the two scoring branches: larger partitions receive proportionally more selections, while smaller partitions still receive explicit coverage\. Thus, the near\-balanced\\hohemasses observed in the experiments are an empirical property of the datasets, not a requirement of the method\.

## 4Experiments

We evaluate post\-generation curation under two settings\. The*general*setting fixes a generator and a synthetic pool, applies each selection method under the same budget, and trains downstream classifiers for in\-domain \(Sec\.[4\.2](https://arxiv.org/html/2607.02637#S4.SS2)\) and out\-of\-domain \(OOD\) \(Sec\.[4\.3](https://arxiv.org/html/2607.02637#S4.SS3)\) evaluation\. The*plug\-in*setting \(Sec\.[4\.4](https://arxiv.org/html/2607.02637#S4.SS4)\) applies the same curation step after task\-customized generation to test whether selection remains useful when the generator has already been adapted to a downstream task\.

### 4\.1Datasets and Baselines

Datasets\.The main evaluation spans classification and segmentation datasets with increasing scale and visual complexity\. For SVHN\[[42](https://arxiv.org/html/2607.02637#bib.bib12)\], CIFAR\-10\[[35](https://arxiv.org/html/2607.02637#bib.bib1)\], and Tiny\-ImageNet\[[37](https://arxiv.org/html/2607.02637#bib.bib9)\], covering 10, 10, and 200 classes respectively, we curate synthetic pools generated by EDM\[[65](https://arxiv.org/html/2607.02637#bib.bib89)\]\. For large\-scale validation, we use ImageNet\-1K \(IN\-1K\) synthetics generated by EDM2\[[30](https://arxiv.org/html/2607.02637#bib.bib5),[29](https://arxiv.org/html/2607.02637#bib.bib6)\]\. In total, the evaluation covers 4 in\-domain benchmarks and 9 OOD benchmarks\. The plug\-in experiments further use task\-customized generation for ImageNet\-100 \(IN\-100\)\[[58](https://arxiv.org/html/2607.02637#bib.bib20)\]and VOCaug\[[16](https://arxiv.org/html/2607.02637#bib.bib22)\]\(Sec\.[4\.4](https://arxiv.org/html/2607.02637#S4.SS4)\)\.

Baselines\.For each comparison, all selection methods operate on the same synthetic pool and use the same selection budget\. In the main evaluation, we compare three categories of selection methods\.RandSelect\[[67](https://arxiv.org/html/2607.02637#bib.bib37),[6](https://arxiv.org/html/2607.02637#bib.bib70),[32](https://arxiv.org/html/2607.02637#bib.bib71)\]is an uninformed random baseline\.CLIP\-Align\[[23](https://arxiv.org/html/2607.02637#bib.bib61)\]represents image–label alignment by ranking generated images with CLIP text–image scores\.RealScore\[[36](https://arxiv.org/html/2607.02637#bib.bib14)\]andSBSim\[[40](https://arxiv.org/html/2607.02637#bib.bib56)\]represent image–image alignment\. In the plug\-in setting, the baselines are generation\-side interventions that already target the downstream task: inference\-time intervention\[[3](https://arxiv.org/html/2607.02637#bib.bib26)\]and the fine\-tuned generatorJoDiffusion\[[62](https://arxiv.org/html/2607.02637#bib.bib27)\]\.

Implementation\.Unless otherwise stated, we useα=0\.5\\alpha=0\.5for all main experiments and keep the downstream training recipe fixed within each comparison\.

![Refer to caption](https://arxiv.org/html/2607.02637v1/x6.png)Figure 6:Average test accuracy over6 runsfor models trained with different amounts of selected synthetic data\. We compare withRandSelect,SBSim, andRealScore;CLIP\-Alignis omitted for readability, with full results in Appendix Tabs\.[10](https://arxiv.org/html/2607.02637#A5.T10),[11](https://arxiv.org/html/2607.02637#A5.T11), and[12](https://arxiv.org/html/2607.02637#A5.T12)\. The black dashed line denotes the corresponding real\-data baseline\.
### 4\.2Scaling Model Performance with Selective Synthetic Data

We first ask whether curation improves the value of synthetic data as the training budget grows\. For each selected set, we train the same downstream architecture with the same recipe and evaluate on the corresponding real test set\. The small\- and medium\-scale experiments use ResNet\-18/50\[[22](https://arxiv.org/html/2607.02637#bib.bib72)\]and EfficientNet\-B0\[[53](https://arxiv.org/html/2607.02637#bib.bib75)\]; the IN\-1K experiments further include ViT\-B/16\[[68](https://arxiv.org/html/2607.02637#bib.bib96)\]\. Full training details are provided in Appendix[E\.2](https://arxiv.org/html/2607.02637#A5.SS2)\.

Table 1:ImageNet\-1K Top\-1 Acc\. \(%\) with 1M and 3M selected samples\. Background colors indicate different models:ViT\-B/16andResNet\-50, evaluated under bothfrom\-scratchandfine\-tuningsettings\. The corresponding training recipes are provided in Appendix[E\.2](https://arxiv.org/html/2607.02637#A5.SS2)\.SelectionScratchFine\-tune\\cellcolorvitblue\+0\\cellcolorvitblue\+1M\\cellcolorvitblue\+3M\\cellcolorresnetorange\+ 0\\cellcolorresnetorange\+1M\\cellcolorresnetorange\+3M\\cellcolorvitblue1M\\cellcolorvitblue3M\\cellcolorresnetorange1M\\cellcolorresnetorange3MRandom64\.2270\.9073\.5469\.2772\.0372\.1378\.5278\.6871\.9073\.65RealScore\-70\.9273\.53\-72\.0172\.1678\.5278\.6771\.9373\.62CLIP\-Align\-69\.1870\.88\-69\.4470\.9872\.4075\.2362\.5767\.57SBSim \(α=0\\alpha=0\)\-70\.9573\.23\-71\.9772\.1878\.5878\.7672\.1874\.14Diversity \(α=1\\alpha=1\)\-71\.4273\.79\-71\.8073\.2578\.7279\.1472\.2574\.25Ours\-71\.4974\.02\-72\.0973\.3378\.9779\.3673\.1474\.76AcrossSVHN,CIFAR\-10, andTiny\-ImageNet, Fig\.[6](https://arxiv.org/html/2607.02637#S4.F6)shows consistent gains from our curation, with the largest benefits appearing as the selection budget grows\. OnSVHN, our method matches the real\-data baseline with substantially fewer examples \(\\ie100 k synthetic samples\), whereas alternative methods require over 200 k to reach comparable accuracy\. OnCIFAR\-10, our method achieves 95% accuracy with 300 k samples, whileRandSelectandRealScorerequire around 500 k, andSBSimdemands substantially more\. This pattern is consistent with the failure mode targeted by our method: once high\-fidelity canonical samples saturate, additional gains depend on selecting less redundant variation\. For the harderTiny\-ImageNet, we use selected synthetic data as augmentation since synthetic\-only training suffers a larger domain gap; our curation improves ResNet\-50 accuracy at every augmentation budget\.

ForIN\-1K, prior work\[[49](https://arxiv.org/html/2607.02637#bib.bib35),[4](https://arxiv.org/html/2607.02637#bib.bib38),[17](https://arxiv.org/html/2607.02637#bib.bib69)\]reported a persistent gap between real and synthetic data at ImageNet scale\. We therefore evaluate two practical uses of selected synthetic data: 1\) training models from scratch with synthetic data as augmentation and 2\) fine\-tuning pre\-trained models using synthetic data alone\. Tab\.[1](https://arxiv.org/html/2607.02637#S4.T1)reports results for ViT\-B/16 and ResNet\-50 with 1M and 3M selected images from an EDM2 pool\. Using the same training configuration, our method gives the best accuracy across all settings, indicating that the fidelity–diversity balance remains useful at ImageNet scale\.

![Refer to caption](https://arxiv.org/html/2607.02637v1/x7.png)Figure 7:OOD Evaluation\. Among different settings, ours consistently achieves the best performance\.
### 4\.3Enhancing Model Generalizability with Selected Samples

Models trained on informative and diverse data are expected to achieve greater robustness under OOD settings\. In this part, we evaluate the generalizability of the best models trained \(from scratch\) in the previous section with OOD datasets\. The results demonstrate that curated synthetic data improves model generalizability\.

OOD of Tiny\-ImageNet\.We use Tiny\-ImageNet\-C\[[25](https://arxiv.org/html/2607.02637#bib.bib11)\], which incorporates various types of corruption\. We classify them into three types: color\-variation set \(\\iebrightness adjustment, contrast variation\), noise\-variation set \(\\iepixelation, Gaussian noise, motion blur\), and compression\-variation set \(\\ieJPEG compression\)\. The results are reported in Fig\.[7](https://arxiv.org/html/2607.02637#S4.F7)\(a\)\.OOD of IN\-1K\.We use several OOD datasets and categorize them into two groups 1\)original suite, including ImageNet\-V2\[[47](https://arxiv.org/html/2607.02637#bib.bib93)\]and ImageNet\-Sketch\[[61](https://arxiv.org/html/2607.02637#bib.bib99)\]; 2\)derivative suite, including ImageNet\-C\[[26](https://arxiv.org/html/2607.02637#bib.bib98)\], \-Drawing, and \-Cartoon\[[48](https://arxiv.org/html/2607.02637#bib.bib95)\], derived from the validation set \(Appendix[15](https://arxiv.org/html/2607.02637#A5.T15)for details\)\. The evaluation results are summarized in Fig\.[7](https://arxiv.org/html/2607.02637#S4.F7)\(b\), \(c\)\.OOD of CIFAR\-10\.CIFAR\-10\-Warehouse\[[52](https://arxiv.org/html/2607.02637#bib.bib76)\]is used as a benchmark, collecting data from diverse sources \(\\egdifferent search engines\)\. The results are reported in Fig\.[7](https://arxiv.org/html/2607.02637#S4.F7)\(d\)\. More results \(\\ieOOD of SVHN, examples of OOD data\) are presented in Appendix[E\.4](https://arxiv.org/html/2607.02637#A5.SS4)\.

### 4\.4Plug\-in Curation after Generator\-Side Interventions

We next test whether curation remains useful after the synthetic pool has already been improved by a generator\-side intervention\. Keeping each task\-adapted pool fixed, we apply our selection as a plug\-in post\-processing step\. We study two representative intervention types: inference\-time generator intervention for classification\[[3](https://arxiv.org/html/2607.02637#bib.bib26)\]and generator fine\-tuning for segmentation\[[62](https://arxiv.org/html/2607.02637#bib.bib27)\]\.

Table 2:Our curation achieves comparable or better accuracy with fewer synthetic samples on IN\-100 using ResNet\-18\.
Table 3:Post\-generation curation improves DeepLabV3\+ performance in mIoU across synthetic data sizes\.

Classification\.We apply our curation to synthetic images generated with inference\-time intervention\[[3](https://arxiv.org/html/2607.02637#bib.bib26)\]on IN\-100, then train ResNet\-18 on the selected data\. As shown in Tab\.[3](https://arxiv.org/html/2607.02637#S4.T3), our curation further improves synthetic data utility: curation provides both an equal\-budget gain and a 40–50% data reduction on task\-aware samples\.

Segmentation\.We curate synthetic VOCaug data produced by the fine\-tuned generator JoDiffusion\[[62](https://arxiv.org/html/2607.02637#bib.bib27)\], then train the downstream segmentation model \(\\ieDeepLabV3\[[9](https://arxiv.org/html/2607.02637#bib.bib36)\]\)\. In Tab\.[3](https://arxiv.org/html/2607.02637#S4.T3), we report mIoU, showing that our method provides additional gains on top of customized generation via fine\-tuning\. Overall, both results suggest that post\-generation curation is a generator\-agnostic approach that further improves the utility of optimized synthetic data, highlighting the practical value of our curation strategy\.

## 5Analysis and Discussion

![Refer to caption](https://arxiv.org/html/2607.02637v1/x8.png)
![Refer to caption](https://arxiv.org/html/2607.02637v1/x9.png)\(a\)StyleGAN\[[28](https://arxiv.org/html/2607.02637#bib.bib97)\]
\(FID: 3\.17\)
![Refer to caption](https://arxiv.org/html/2607.02637v1/x10.png)\(b\)EDM\[[27](https://arxiv.org/html/2607.02637#bib.bib42)\]
\(FID: 2\.67\)
![Refer to caption](https://arxiv.org/html/2607.02637v1/x11.png)\(c\)EDM2\[[30](https://arxiv.org/html/2607.02637#bib.bib5)\]
\(FID: 1\.39\)

Figure 8:Testing accuracy versus trade\-offα\\alpha\(0 = fidelity, 1 = diversity\)\. Each subfigure compares the 100K, 300K, and 500K settings with trend lines, and the optimal point on each curve is marked by a star\. Results are reported withResNet\-18using theaverage accuracy over 8 runs\.CLIP Filtering Biases Toward Canonical Modes\.The weak performance ofCLIP\-Alignin Sec\.[4\.2](https://arxiv.org/html/2607.02637#S4.SS2)suggests that image–text alignment alone is not a sufficient selection criterion\. Tab\.[4](https://arxiv.org/html/2607.02637#S5.T4)provides a quantitative diagnostic on CIFAR\-10: High\-CLIP\-score samples are consistently closer toℐH​O\\mathcal\{I\}\_\{HO\}than toℐH​E\\mathcal\{I\}\_\{HE\}across CLIP, MoCo, and ConvNeXt features\. This supports the interpretation that CLIP filtering over\-selects canonical modes and under\-covers harder variations\.

Generator Quality and Data Budget Shift the Optimal Fidelity\-Diversity Balance \(α\\bm\{\\alpha\}\)\.We explore the relationship betweenα\\alphain Eq\.[6](https://arxiv.org/html/2607.02637#S3.E6)and different GMs\. Our experiments use 1M synthetic CIFAR\-10 samples from three GMs\. As shown in Fig\.[8](https://arxiv.org/html/2607.02637#S5.F8), the optimalα\\alphadepends on both the quality of the synthetic data and the volume of training data\. \(1\) For StyleGAN2 \(Fig\.[8\(a\)](https://arxiv.org/html/2607.02637#S5.F8.sf1)\), which yields the highest FID, fidelity remains the key factor for classification performance; \(2\) When the generated distributions are closer to the real \(as in EDM and EDM2\), Figs\.[8\(b\)](https://arxiv.org/html/2607.02637#S5.F8.sf2),[8\(c\)](https://arxiv.org/html/2607.02637#S5.F8.sf3)demonstrate the benefit of tuning the trade\-off factorα\\alpha\. \(3\) Arcoss all the datasets, as the training volume increases, the optimalα\\alphashifts toward larger values, underscoring the increasing importance of diversity at larger scales, where canonical high\-fidelity samples provide limited additional gains\. \(4\) In terms of the bestα\\alphain each GMs, the biggest shift is observed in the most advanced EDM2, suggesting that with larger training volumes and highly realistic generations, diversity becomes the decisive factor\.

![Refer to caption](https://arxiv.org/html/2607.02637v1/x12.png)Figure 9:Testing accuracy of models trained on selected synthetic data with different feature extractors\. The corresponding training configurations are provided in Appendix[F\.2](https://arxiv.org/html/2607.02637#A6.SS2)\.Impact of Feature Extractors\.Since our strategy relies on image features, we examine whether the choice of feature extractor matters\. We conduct an ablation study using three representative encoders, each reflecting a distinct training paradigm: SigLIP \(text–image alignment\)\[[71](https://arxiv.org/html/2607.02637#bib.bib92)\], DINOv3 \(self\-supervised\)\[[51](https://arxiv.org/html/2607.02637#bib.bib91)\], and ViT \(supervised\)\. Evaluations are performed on two datasets with different image resolutions, CIFAR\-10 and ImageNet\-100 \(configurations are summarized in Appendix[F\.2](https://arxiv.org/html/2607.02637#A6.SS2)\)\. The results, summarized in Fig\.[9](https://arxiv.org/html/2607.02637#S5.F9), show that although SigLIP achieves the best performance, the absolute differences among extractors remain modest\. This suggests that SigLIP is a promising choice, but its impact is relatively minor compared to the quality of synthetic data and the choice ofα\\alphain the proposed strategy\.

Curation Overhead Is Small Relative to Training\.Comparing to previous post\-generation selection, our curation introduces unique cost inℐH​O/ℐH​E\\mathcal\{I\}\_\{HO\}/\\mathcal\{I\}\_\{HE\}splitting, scoring and selection\. Once features are extracted, all such operations reduce to batched cosine similarity computations that are GPU\-friendly and fast in practice \(implementation in Appendix[G\.1](https://arxiv.org/html/2607.02637#A7.SS1),[G\.2](https://arxiv.org/html/2607.02637#A7.SS2)\)\. Tab\.[5](https://arxiv.org/html/2607.02637#S5.T5)shows the runtime of each stage of our curation for 10M synthetic samples across varying real\-data scales\. Feature extraction dominates the overall overhead\. Since it is a one\-time inference cost shared by most instance\-selection baselines, the overhead specific to our method remains negligible even at large scales\.

Table 4:Similarity of high\-CLIP data toℐH​O/ℐH​E\\mathcal\{I\}\_\{HO\}/\\mathcal\{I\}\_\{HE\}\.
Table 5:Curation steps overhead across real data sizes\.
Table 6:Compute–accuracy trade\-off in Eff\.Net\-B0 \(1×\\timesNVIDIA L40S\)\.

Furthermore, when downstream training is considered, the extra curation cost is well justified\. In the Tiny\-ImageNet setting, curation of a 1M\-image pool takes 3\.8 minutes in total \(3\.5 minutes for feature extraction and 0\.3 minutes for scoring and selection\)\. In contrast, training on 200K, 300K, and 500K augmented samples takes 80, 120, and 210 minutes, respectively\. As shown in Tab \.[6](https://arxiv.org/html/2607.02637#S5.T6), despite incurring 3\.8 additional minutes over random selection, our method delivers better performance with fewer training samples, effectively compensating for the curation overhead in downstream training\.

The Characteristics of\\hoheSplit in Real Data\.Splittingℐℋ​𝒪\\mathcal\{I\_\{HO\}\}andℐℋ​ℰ\\mathcal\{I\_\{HE\}\}within a class across datasets, the number of samples in each subset remains nearly equal, suggesting that*approximately half of the samples effectively capture the primary semantics of the class\.*Usingℐℋ​𝒪\\mathcal\{I\_\{HO\}\}andℐℋ​ℰ\\mathcal\{I\_\{HE\}\}as curation references, we observed that although the two sets have comparable volumes,ℐℋ​𝒪\\mathcal\{I\_\{HO\}\}yields fewer unique synthetic instances\. We attribute this to the*higher intra\-class similarity withinℐℋ​𝒪\\mathcal\{I\_\{HO\}\}\(Tab\.[7](https://arxiv.org/html/2607.02637#A1.T7)in the Appendix\), which makes it more challenging for GMs to capture fine\-grained variations among similar concepts, compared to the more diverseℐℋ​ℰ\\mathcal\{I\_\{HE\}\}*\.

## 6Conclusions

We introduced a generator\-agnostic framework for post\-generation curation of synthetic images\. By splitting real data into\\hoand\\hereferences and selecting synthetics with a partition\-wise fidelity–diversity score, the method improves in\-domain accuracy, strengthens OOD generalization, reduces the amount of synthetic data needed to match real\-data baselines, and remains complementary to task\-customized generation for both classification and segmentation\. The analysis further suggests a practical principle for synthetic\-data use: as generator fidelity and training budgets increase, curation should shift from merely retaining recognizable samples toward allocating more budget to non\-redundant variation\.

## References

- \[1\]A\. Abbas, E\. Rusak, K\. Tirumala, W\. Brendel, K\. Chaudhuri, and A\. S\. Morcos\(2024\)Effective pruning of web\-scale datasets based on complexity of concept clusters\.arXiv preprint arXiv:2401\.04578\.Cited by:[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p1.1)\.
- \[2\]A\. Abbas, K\. Tirumala, D\. Simig, S\. Ganguli, and A\. S\. Morcos\(2023\)Semdedup: data\-efficient learning at web\-scale through semantic deduplication\.arXiv preprint arXiv:2303\.09540\.Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p3.1),[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p1.1)\.
- \[3\]R\. Askari Hemmat, M\. Hall, A\. Sun, C\. Ross, M\. Drozdzal, and A\. Romero\-Soriano\(2024\)Improving geo\-diversity of generated images with contextualized vendi score guidance\.InEuropean Conference on Computer Vision,pp\. 213–229\.Cited by:[§E\.1](https://arxiv.org/html/2607.02637#A5.SS1.p1.1),[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p2.1),[§4\.4](https://arxiv.org/html/2607.02637#S4.SS4.p1.1),[§4\.4](https://arxiv.org/html/2607.02637#S4.SS4.p2.1),[Table 3](https://arxiv.org/html/2607.02637#S4.T3.1.1.1.1.1),[Table 3](https://arxiv.org/html/2607.02637#S4.T3.2.2.2.2.1)\.
- \[4\]S\. Azizi, S\. Kornblith, C\. Saharia, M\. Norouzi, and D\. J\. Fleet\(2023\)Synthetic data from diffusion models improves imagenet classification\.Transactions on Machine Learning Research\.Note:External Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=DlRsoxjyPm)Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p1.1),[§1](https://arxiv.org/html/2607.02637#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p2.1),[§4\.2](https://arxiv.org/html/2607.02637#S4.SS2.p3.1)\.
- \[5\]B\. Bhattarai, S\. Baek, R\. Bodur, and T\. Kim\(2020\-05\)Sampling strategies for gan synthetic data\.InICASSP 2020 \- 2020 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),pp\. 2303–2307\.External Links:[Link](http://dx.doi.org/10.1109/ICASSP40776.2020.9054677),[Document](https://dx.doi.org/10.1109/icassp40776.2020.9054677)Cited by:[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p2.1)\.
- \[6\]N\. Carlini, J\. Hayes, M\. Nasr, M\. Jagielski, V\. Sehwag, F\. Tramer, B\. Balle, D\. Ippolito, and E\. Wallace\(2023\)Extracting training data from diffusion models\.In32nd USENIX Security Symposium \(USENIX Security 23\),pp\. 5253–5270\.Cited by:[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p2.1)\.
- \[7\]S\. Chandhok, Q\. Yang, O\. Mañas, K\. Jain, L\. Sigal, and A\. Agrawal\(2025\)Learning what matters: prioritized concept learning via relative error\-driven sample selection\.arXiv preprint arXiv:2506\.01085\.Cited by:[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p1.1)\.
- \[8\]D\. Chen, X\. Qi, Y\. Zheng, Y\. Lu, Y\. Huang, and Z\. Li\(2023\)Deep data augmentation for weed recognition enhancement: a diffusion probabilistic model and transfer learning based approach\.In2023 ASABE Annual International Meeting,pp\. 1\.Cited by:[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p2.1)\.
- \[9\]L\. Chen, Y\. Zhu, G\. Papandreou, F\. Schroff, and H\. Adam\(2018\)Encoder\-decoder with atrous separable convolution for semantic image segmentation\.InProceedings of the European conference on computer vision \(ECCV\),pp\. 801–818\.Cited by:[§4\.4](https://arxiv.org/html/2607.02637#S4.SS4.p3.1)\.
- \[10\]Y\. Chen, Z\. Yan, Y\. Zhu, Z\. Ren, J\. Shen, and Y\. Huang\(2023\)Data augmentation for environmental sound classification using diffusion probabilistic model with top\-k selection discriminator\.InInternational Conference on Intelligent Computing,pp\. 283–295\.Cited by:[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p2.1)\.
- \[11\]Y\. Chen, Y\. Zhu, Z\. Yan, Z\. Ren, Y\. Huang, J\. Shen, and L\. Chen\(2023\)Effective audio classification network based on paired inverse pyramid structure and dense mlp block\.InInternational Conference on Intelligent Computing,pp\. 70–84\.Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p4.1)\.
- \[12\]X\. Chen\*, S\. Xie\*, and K\. He\(2021\)An empirical study of training self\-supervised vision transformers\.arXiv preprint arXiv:2104\.02057\.Cited by:[§3\.1](https://arxiv.org/html/2607.02637#S3.SS1.p3.5)\.
- \[13\]V\. G\. T\. da Costa, N\. Dall’Asen, Y\. Wang, N\. Sebe, and E\. Ricci\(2023\)Diversified in\-domain synthesis with efficient fine\-tuning for few\-shot classification\.arXiv preprint arXiv:2312\.03046\.Cited by:[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p3.1)\.
- \[14\]N\. Dall’Asen, X\. Zhang, R\. A\. Hemmat, M\. Hall, J\. Verbeek, A\. Romero\-Soriano, and M\. Drozdzal\(2025\)Increasing the utility of synthetic images through chamfer guidance\.arXiv preprint arXiv:2508\.10631\.Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p3.1)\.
- \[15\]L\. Dunlap, A\. Umino, H\. Zhang, J\. Yang, J\. E\. Gonzalez, and T\. Darrell\(2023\)Diversify your vision datasets with automatic diffusion\-based augmentation\.InThirty\-seventh Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=9wrYfqdrwk)Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p3.1),[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p3.1)\.
- \[16\]M\. Everingham, S\. M\. A\. Eslami, L\. Van Gool, C\. K\. I\. Williams, J\. Winn, and A\. Zisserman\(2015\)The pascal visual object classes challenge: a retrospective\.International Journal of Computer Vision111\(1\),pp\. 98–136\.Cited by:[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p1.1)\.
- \[17\]L\. Fan, K\. Chen, D\. Krishnan, D\. Katabi, P\. Isola, and Y\. Tian\(2024\)Scaling laws of synthetic images for model training … for now\.InCVPR,pp\. 7382–7392\.External Links:[Link](https://doi.org/10.1109/CVPR52733.2024.00705)Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p1.1),[§1](https://arxiv.org/html/2607.02637#S1.p4.1),[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1),[§4\.2](https://arxiv.org/html/2607.02637#S4.SS2.p3.1)\.
- \[18\]S\. Y\. Gadre, G\. Ilharco, A\. Fang, J\. Hayase, G\. Smyrnis, T\. Nguyen, R\. Marten, M\. Wortsman, D\. Ghosh, J\. Zhang, E\. Orgad, R\. Entezari, G\. Daras, S\. M\. Pratt, V\. Ramanujan, Y\. Bitton, K\. Marathe, S\. Mussmann, R\. Vencu, M\. Cherti, R\. Krishna, P\. W\. Koh, O\. Saukh, A\. Ratner, S\. Song, H\. Hajishirzi, A\. Farhadi, R\. Beaumont, S\. Oh, A\. Dimakis, J\. Jitsev, Y\. Carmon, V\. Shankar, and L\. Schmidt\(2023\)DataComp: in search of the next generation of multimodal datasets\.InThirty\-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track,External Links:[Link](https://openreview.net/forum?id=dVaWCDMBof)Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p3.1)\.
- \[19\]J\. Guo, S\. Yang, Y\. Huang, Y\. Long, X\. Xia, X\. Su, B\. Zhao, Z\. Xie, and L\. Nie\(2026\)UtilGen: utility\-centric generative data augmentation with dual\-level task adaptation\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=tBQEPRFT60)Cited by:[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1)\.
- \[20\]H\. A\. A\. K\. Hammoud, H\. Itani, F\. Pizzati, P\. Torr, A\. Bibi, and B\. Ghanem\(2024\)SynthCLIP: are we ready for a fully synthetic clip training?\.External Links:2402\.01832,[Link](https://arxiv.org/abs/2402.01832)Cited by:[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1)\.
- \[21\]R\. Hataya, H\. Bao, and H\. Arai\(2023\-10\)Will large\-scale generative models corrupt future datasets?\.In2023 IEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 20498–20508\.External Links:[Link](http://dx.doi.org/10.1109/ICCV51070.2023.01879),[Document](https://dx.doi.org/10.1109/iccv51070.2023.01879)Cited by:[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1)\.
- \[22\]K\. He, X\. Zhang, S\. Ren, and J\. Sun\(2016\)Deep residual learning for image recognition\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\. 770–778\.Cited by:[§4\.2](https://arxiv.org/html/2607.02637#S4.SS2.p1.1)\.
- \[23\]R\. He, S\. Sun, X\. Yu, C\. Xue, W\. Zhang, P\. Torr, S\. Bai, and X\. Qi\(2023\)Is synthetic data from generative models ready for image recognition?\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=nUmCcZ5RKF)Cited by:[Table 10](https://arxiv.org/html/2607.02637#A5.T10.13.13.13.7),[Table 12](https://arxiv.org/html/2607.02637#A5.T12.19.17.17.9),[Table 12](https://arxiv.org/html/2607.02637#A5.T12.70.68.68.9),[§1](https://arxiv.org/html/2607.02637#S1.p1.1),[§1](https://arxiv.org/html/2607.02637#S1.p4.1),[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p2.1),[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p2.1)\.
- \[24\]R\. A\. Hemmat, M\. Pezeshki, F\. Bordes, M\. Drozdzal, and A\. Romero\-Soriano\(2024\)Feedback\-guided data synthesis for imbalanced classification\.External Links:2310\.00158,[Link](https://arxiv.org/abs/2310.00158)Cited by:[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p3.1)\.
- \[25\]D\. Hendrycks and T\. Dietterich\(2019\)Benchmarking neural network robustness to common corruptions and perturbations\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=HJz6tiCqYm)Cited by:[§E\.4](https://arxiv.org/html/2607.02637#A5.SS4.p4.1),[§4\.3](https://arxiv.org/html/2607.02637#S4.SS3.p2.1)\.
- \[26\]D\. Hendrycks and T\. Dietterich\(2019\)Benchmarking neural network robustness to common corruptions and perturbations\.Proceedings of the International Conference on Learning Representations\.Cited by:[§E\.4](https://arxiv.org/html/2607.02637#A5.SS4.p8.1),[§4\.3](https://arxiv.org/html/2607.02637#S4.SS3.p2.1)\.
- \[27\]T\. Karras, M\. Aittala, T\. Aila, and S\. Laine\(2022\)Elucidating the design space of diffusion\-based generative models\.InProc\. NeurIPS,Cited by:[Appendix C](https://arxiv.org/html/2607.02637#A3.p2.5),[8\(b\)](https://arxiv.org/html/2607.02637#S5.F8.sf2)\.
- \[28\]T\. Karras, M\. Aittala, J\. Hellsten, S\. Laine, J\. Lehtinen, and T\. Aila\(2020\)Training generative adversarial networks with limited data\.InProc\. NeurIPS,Cited by:[8\(a\)](https://arxiv.org/html/2607.02637#S5.F8.sf1)\.
- \[29\]T\. Karras, M\. Aittala, T\. Kynkäänniemi, J\. Lehtinen, T\. Aila, and S\. Laine\(2024\)Guiding a diffusion model with a bad version of itself\.InProc\. NeurIPS,Cited by:[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p1.1)\.
- \[30\]T\. Karras, M\. Aittala, J\. Lehtinen, J\. Hellsten, T\. Aila, and S\. Laine\(2024\)Analyzing and improving the training dynamics of diffusion models\.InProc\. CVPR,Cited by:[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p1.1),[8\(c\)](https://arxiv.org/html/2607.02637#S5.F8.sf3)\.
- \[31\]J\. M\. Kim, J\. Bader, S\. Alaniz, C\. Schmid, and Z\. Akata\(2024\)Datadream: few\-shot guided dataset generation\.InEuropean Conference on Computer Vision,pp\. 252–268\.Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p2.1)\.
- \[32\]M\. Kim, F\. Liu, A\. Jain, and X\. Liu\(2023\)Dcface: synthetic face generation with dual condition diffusion model\.InProceedings of the ieee/cvf conference on computer vision and pattern recognition,pp\. 12715–12725\.Cited by:[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p2.1)\.
- \[33\]S\. Kim, M\. Kim, and D\. Park\(2025\)Test\-time alignment of diffusion models without reward over\-optimization\.arXiv preprint arXiv:2501\.05803\.Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p2.1)\.
- \[34\]M\. Kirchhof, J\. Thornton, L\. Béthune, P\. Ablin, E\. Ndiaye, and M\. Cuturi\(2024\)Shielded diffusion: generating novel and diverse images using sparse repellency\.arXiv preprint arXiv:2410\.06025\.Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p3.1)\.
- \[35\]A\. Krizhevsky, V\. Nair, and G\. Hinton\(2010\)Cifar\-10 \(canadian institute for advanced research\)\.URL http://www\. cs\. toronto\. edu/kriz/cifar\. html5\(4\),pp\. 1\.Cited by:[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p1.1)\.
- \[36\]T\. Kynkäänniemi, T\. Karras, S\. Laine, J\. Lehtinen, and T\. Aila\(2019\)Improved precision and recall metric for assessing generative models\.InAdvances in Neural Information Processing Systems,H\. Wallach, H\. Larochelle, A\. Beygelzimer, F\. d'Alché\-Buc, E\. Fox, and R\. Garnett \(Eds\.\),Vol\.32,pp\.\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2019/file/0234c510bc6d908b28c70ff313743079-Paper.pdf)Cited by:[Table 10](https://arxiv.org/html/2607.02637#A5.T10.19.19.19.7),[Table 12](https://arxiv.org/html/2607.02637#A5.T12.27.25.25.9),[Table 12](https://arxiv.org/html/2607.02637#A5.T12.78.76.76.9),[§1](https://arxiv.org/html/2607.02637#S1.p4.1),[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p2.1),[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p2.1)\.
- \[37\]Y\. Le and X\. S\. Yang\(2015\)Tiny imagenet visual recognition challenge\.External Links:[Link](https://api.semanticscholar.org/CorpusID:16664790)Cited by:[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p1.1)\.
- \[38\]Y\. Li, X\. Dong, C\. Chen, J\. Li, Y\. Wen, M\. Spranger, and L\. Lyu\(2024\)IS SYNTHETIC DATA USEFUL FOR TRANSFER LEARNING? AN INVESTIGATION INTO DATA GENERATION, VOLUME, AND UTILIZATION\.External Links:[Link](https://openreview.net/forum?id=CjPt1AC6w0)Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1)\.
- \[39\]Y\. Liang, S\. Bhardwaj, and T\. Zhou\(2025\)Diffusion curriculum: synthetic\-to\-real data curriculum via image\-guided diffusion\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 1697–1707\.Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p2.1)\.
- \[40\]S\. Lin, K\. Wang, X\. Zeng, and R\. Zhao\(2023\)Explore the power of synthetic data on few\-shot object detection\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 638–647\.Cited by:[Table 10](https://arxiv.org/html/2607.02637#A5.T10.25.25.25.7),[Table 12](https://arxiv.org/html/2607.02637#A5.T12.28.26.26.1),[Table 12](https://arxiv.org/html/2607.02637#A5.T12.79.77.77.1),[§1](https://arxiv.org/html/2607.02637#S1.p4.1),[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p2.1),[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p2.1)\.
- \[41\]M\. F\. Naeem, S\. J\. Oh, Y\. Uh, Y\. Choi, and J\. Yoo\(2020\)Reliable fidelity and diversity metrics for generative models\.External Links:2002\.09797Cited by:[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p2.1)\.
- \[42\]Y\. Netzer, T\. Wang, A\. Coates, A\. Bissacco, B\. Wu, and A\. Y\. Ng\(2011\)Reading digits in natural images with unsupervised feature learning\.InNIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011,External Links:[Link](http://ufldl.stanford.edu/housenumbers/nips2011_housenumbers.pdf)Cited by:[§E\.4](https://arxiv.org/html/2607.02637#A5.SS4.p2.1),[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p1.1)\.
- \[43\]K\. Qiu, Z\. Gao, Z\. Zhou, M\. Sun, and Y\. Guo\(2025\)Noise\-consistent siamese\-diffusion for medical image synthesis and segmentation\.InProceedings of the Computer Vision and Pattern Recognition Conference,pp\. 15672–15681\.Cited by:[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1)\.
- \[44\]M\. Qraitem, K\. Saenko, and B\. A\. Plummer\(2024\)From fake to real: pretraining on balanced synthetic images to prevent spurious correlations in image recognition\.External Links:2308\.04553,[Link](https://arxiv.org/abs/2308.04553)Cited by:[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1)\.
- \[45\]A\. Radford, J\. W\. Kim, C\. Hallacy, A\. Ramesh, G\. Goh, S\. Agarwal, G\. Sastry, A\. Askell, P\. Mishkin, J\. Clark, G\. Krueger, and I\. Sutskever\(2021\-18–24 Jul\)Learning transferable visual models from natural language supervision\.InProceedings of the 38th International Conference on Machine Learning,M\. Meila and T\. Zhang \(Eds\.\),Proceedings of Machine Learning Research, Vol\.139,pp\. 8748–8763\.External Links:[Link](https://proceedings.mlr.press/v139/radford21a.html)Cited by:[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p2.1)\.
- \[46\]F\. Rahat, M\. S\. Hossain, M\. R\. Ahmed, S\. K\. Jha, and R\. Ewetz\(2025\)Data augmentation for image classification using generative ai\.In2025 IEEE/CVF Winter Conference on Applications of Computer Vision \(WACV\),pp\. 4173–4182\.Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p2.1)\.
- \[47\]B\. Recht, R\. Roelofs, L\. Schmidt, and V\. Shankar\(2019\)Do imagenet classifiers generalize to imagenet?\.External Links:1902\.10811,[Link](https://arxiv.org/abs/1902.10811)Cited by:[§E\.4](https://arxiv.org/html/2607.02637#A5.SS4.p6.1),[§4\.3](https://arxiv.org/html/2607.02637#S4.SS3.p2.1)\.
- \[48\]T\. Salvador and A\. M\. Oberman\(2022\)ImageNet\-cartoon and imagenet\-drawing: two domain shift datasets for imagenet\.InICML 2022 Shift Happens Workshop,External Links:[Link](https://openreview.net/forum?id=YlAUXhjwaQt)Cited by:[§E\.4](https://arxiv.org/html/2607.02637#A5.SS4.p10.1),[§E\.4](https://arxiv.org/html/2607.02637#A5.SS4.p9.1),[§4\.3](https://arxiv.org/html/2607.02637#S4.SS3.p2.1)\.
- \[49\]M\. B\. Sariyildiz, K\. Alahari, D\. Larlus, and Y\. Kalantidis\(2023\)Fake it till you make it: learning transferable representations from synthetic imagenet clones\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 8011–8021\.Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1),[§4\.2](https://arxiv.org/html/2607.02637#S4.SS2.p3.1)\.
- \[50\]J\. Shipard, A\. Wiliem, K\. N\. Thanh, W\. Xiang, and C\. Fookes\(2023\-06\)Diversity is definitely needed: improving model\-agnostic zero\-shot classification via stable diffusion\.In2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops \(CVPRW\),pp\. 769–778\.External Links:[Link](http://dx.doi.org/10.1109/CVPRW59228.2023.00084),[Document](https://dx.doi.org/10.1109/cvprw59228.2023.00084)Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p3.1),[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p3.1)\.
- \[51\]O\. Siméoni, H\. V\. Vo, M\. Seitzer, F\. Baldassarre, M\. Oquab, C\. Jose, V\. Khalidov, M\. Szafraniec, S\. Yi, M\. Ramamonjisoa, F\. Massa, D\. Haziza, L\. Wehrstedt, J\. Wang, T\. Darcet, T\. Moutakanni, L\. Sentana, C\. Roberts, A\. Vedaldi, J\. Tolan, J\. Brandt, C\. Couprie, J\. Mairal, H\. Jégou, P\. Labatut, and P\. Bojanowski\(2025\)DINOv3\.External Links:2508\.10104Cited by:[§5](https://arxiv.org/html/2607.02637#S5.p3.1)\.
- \[52\]X\. Sun, X\. Leng, Z\. Wang, Y\. Yang, Z\. Huang, and L\. Zheng\(2024\)Cifar\-10\-warehouse: broad and more realistic testbeds in model generalization analysis\.InICLR,Cited by:[§E\.4](https://arxiv.org/html/2607.02637#A5.SS4.p3.1),[§4\.3](https://arxiv.org/html/2607.02637#S4.SS3.p2.1)\.
- \[53\]M\. Tan and Q\. Le\(2019\)Efficientnet: rethinking model scaling for convolutional neural networks\.InInternational conference on machine learning,pp\. 6105–6114\.Cited by:[§4\.2](https://arxiv.org/html/2607.02637#S4.SS2.p1.1)\.
- \[54\]D\. Tang, X\. Cao, X\. Wu, J\. Li, J\. Yao, X\. Bai, D\. Jiang, Y\. Li, and D\. Meng\(2025\)AeroGen: enhancing remote sensing object detection with diffusion\-driven data generation\.InProceedings of the Computer Vision and Pattern Recognition Conference,pp\. 3614–3624\.Cited by:[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1)\.
- \[55\]H\. Tang, S\. Yu, J\. Pang, and B\. Zhang\(2025\)A training\-free synthetic data selection method for semantic segmentation\.External Links:2501\.15201,[Link](https://arxiv.org/abs/2501.15201)Cited by:[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p2.1)\.
- \[56\]Y\. Tian, L\. Fan, K\. Chen, D\. Katabi, D\. Krishnan, and P\. Isola\(2023\)Learning vision from models rivals learning vision from data\.External Links:2312\.17742,[Link](https://arxiv.org/abs/2312.17742)Cited by:[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1)\.
- \[57\]Y\. Tian, L\. Fan, P\. Isola, H\. Chang, and D\. Krishnan\(2023\)StableRep: synthetic images from text\-to\-image models make strong visual representation learners\.External Links:2306\.00984,[Link](https://arxiv.org/abs/2306.00984)Cited by:[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1)\.
- \[58\]Y\. Tian, D\. Krishnan, and P\. Isola\(2020\)Contrastive multiview coding\.InEuropean conference on computer vision,pp\. 776–794\.Cited by:[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p1.1)\.
- \[59\]B\. Trabucco, K\. Doherty, M\. A\. Gurinas, and R\. Salakhutdinov\(2024\)Effective data augmentation with diffusion models\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=ZWzUA9zeAg)Cited by:[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p3.1)\.
- \[60\]T\. Vu, M\. Luong, Q\. Le, G\. Simon, and M\. Iyyer\(2021\)STraTA: self\-training with task augmentation for better few\-shot learning\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](http://dx.doi.org/10.18653/v1/2021.emnlp-main.462),[Document](https://dx.doi.org/10.18653/v1/2021.emnlp-main.462)Cited by:[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p2.1)\.
- \[61\]H\. Wang, S\. Ge, Z\. Lipton, and E\. P\. Xing\(2019\)Learning robust global representations by penalizing local predictive power\.InAdvances in Neural Information Processing Systems,pp\. 10506–10518\.Cited by:[§E\.4](https://arxiv.org/html/2607.02637#A5.SS4.p7.1),[§4\.3](https://arxiv.org/html/2607.02637#S4.SS3.p2.1)\.
- \[62\]H\. Wang, L\. Zhang, W\. Liu, D\. Jiang, W\. Wei, and C\. Ding\(2026\)JoDiffusion: jointly diffusing image with pixel\-level annotations for semantic segmentation promotion\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 9775–9783\.Cited by:[§E\.1](https://arxiv.org/html/2607.02637#A5.SS1.p1.1),[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p2.1),[§4\.4](https://arxiv.org/html/2607.02637#S4.SS4.p1.1),[§4\.4](https://arxiv.org/html/2607.02637#S4.SS4.p3.1),[Table 3](https://arxiv.org/html/2607.02637#S4.T3.fig1.5.1.1.2)\.
- \[63\]Y\. Wang, J\. Zhang, and Y\. Wang\(2024\)Do generated data always help contrastive learning?\.External Links:2403\.12448,[Link](https://arxiv.org/abs/2403.12448)Cited by:[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1)\.
- \[64\]Y\. O\. Wang, Y\. Chung, C\. H\. Wu, and F\. De la Torre\(2024\)Domain gap embeddings for generative dataset augmentation\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 28684–28694\.Cited by:[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1)\.
- \[65\]Z\. Wang, T\. Pang, C\. Du, M\. Lin, W\. Liu, and S\. Yan\(2023\)Better diffusion models further improve adversarial training\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p1.1)\.
- \[66\]R\. Wightman\(2019\)PyTorch image models\.GitHub\.Note:[https://github\.com/rwightman/pytorch\-image\-models](https://github.com/rwightman/pytorch-image-models)External Links:[Document](https://dx.doi.org/10.5281/zenodo.4414861)Cited by:[Table 8](https://arxiv.org/html/2607.02637#A3.T8.4.11.11.3.1.1)\.
- \[67\]E\. Wood, T\. Baltrušaitis, C\. Hewitt, S\. Dziadzio, M\. Johnson, V\. Estellers, T\. J\. Cashman, and J\. Shotton\(2021\)Fake it till you make it: face analysis in the wild using synthetic data alone\.External Links:2109\.15102,[Link](https://arxiv.org/abs/2109.15102)Cited by:[Table 10](https://arxiv.org/html/2607.02637#A5.T10.7.7.7.7),[Table 12](https://arxiv.org/html/2607.02637#A5.T12.11.9.9.9),[Table 12](https://arxiv.org/html/2607.02637#A5.T12.62.60.60.9),[§4\.1](https://arxiv.org/html/2607.02637#S4.SS1.p2.1)\.
- \[68\]B\. Wu, C\. Xu, X\. Dai, A\. Wan, P\. Zhang, Z\. Yan, M\. Tomizuka, J\. Gonzalez, K\. Keutzer, and P\. Vajda\(2020\)Visual transformers: token\-based image representation and processing for computer vision\.External Links:2006\.03677Cited by:[§4\.2](https://arxiv.org/html/2607.02637#S4.SS2.p1.1)\.
- \[69\]J\. Yuan, J\. Zhang, S\. Sun, P\. Torr, and B\. Zhao\(2024\)Real\-fake: effective training data synthesis through distribution matching\.External Links:2310\.10402,[Link](https://arxiv.org/abs/2310.10402)Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p2.1)\.
- \[70\]D\. Zeng, Y\. Wu, X\. Hu, X\. Xu, and Y\. Shi\(2025\)Contrastive learning with synthetic positives\.External Links:2408\.16965,[Link](https://arxiv.org/abs/2408.16965)Cited by:[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1)\.
- \[71\]X\. Zhai, B\. Mustafa, A\. Kolesnikov, and L\. Beyer\(2023\)Sigmoid loss for language image pre\-training\.External Links:2303\.15343Cited by:[§5](https://arxiv.org/html/2607.02637#S5.p3.1)\.
- \[72\]Y\. Zhang, H\. Ling, J\. Gao, K\. Yin, J\. Lafleche, A\. Barriuso, A\. Torralba, and S\. Fidler\(2021\-06\)DatasetGAN: efficient labeled data factory with minimal human effort\.In2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 10140–10150\.External Links:[Link](http://dx.doi.org/10.1109/CVPR46437.2021.01001),[Document](https://dx.doi.org/10.1109/cvpr46437.2021.01001)Cited by:[§2\.2](https://arxiv.org/html/2607.02637#S2.SS2.p2.1)\.
- \[73\]Y\. Zhang, H\. Ling, J\. Gao, K\. Yin, J\. Lafleche, A\. Barriuso, A\. Torralba, and S\. Fidler\(2021\)Datasetgan: efficient labeled data factory with minimal human effort\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 10145–10155\.Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p4.1)\.
- \[74\]C\. Zheng, G\. Wu, and C\. Li\(2023\)Toward understanding generative data augmentation\.InThirty\-seventh Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=W5Clq1bSrR)Cited by:[§1](https://arxiv.org/html/2607.02637#S1.p1.1)\.
- \[75\]Y\. Zhou, H\. Sahak, and J\. Ba\(2023\)Training on thin air: improve image classification with generated data\.External Links:2305\.15316,[Link](https://arxiv.org/abs/2305.15316)Cited by:[§2\.1](https://arxiv.org/html/2607.02637#S2.SS1.p1.1)\.

Appendix

The supplementary material is organized by the role each section plays in supporting the main paper\. Appendix[A](https://arxiv.org/html/2607.02637#A1)first gives visual examples and intra\-class statistics that characterize the\\ho/\\hepartition\. Appendix[B](https://arxiv.org/html/2607.02637#A2)explains the motivation and theoretical interpretation of the partition\. Appendix[12](https://arxiv.org/html/2607.02637#A3.F12)examines how the\\ho/\\hesplit exposes generator and discriminator biases\. Appendix[D](https://arxiv.org/html/2607.02637#A4)describes the synthetic\-data selection procedure and visualizes the diversity score\. Appendix[E](https://arxiv.org/html/2607.02637#A5)provides the full experimental details, including compute resources, generator usage, training recipes, in\-domain results, and out\-of\-domain results\. Appendix[F](https://arxiv.org/html/2607.02637#A6)reports additional ablations, Appendix[G](https://arxiv.org/html/2607.02637#A7)gives the scoring and selection implementation, and Appendices[H](https://arxiv.org/html/2607.02637#A8)–[I](https://arxiv.org/html/2607.02637#A9)discuss limitations and broader impacts\.

## Appendix AVisual and Statistical Properties of the HO/HE Partition

![[Uncaptioned image]](https://arxiv.org/html/2607.02637v1/x13.png)

Figure 10:For CIFAR\-10, we select both\\hoand\\heimages within each class\.\\hoinstances \(framed in blue\) are more representative and better express the core semantics of given class\.\\heinstances \(framed in green\) are more diverse, capturing a broader range of variations
This section provides the visual and quantitative evidence referenced in Sec\.[3\.1](https://arxiv.org/html/2607.02637#S3.SS1)\. We partition each real class intoℐH​O\\mathcal\{I\}\_\{HO\}, which contains local canonical representatives, andℐH​E\\mathcal\{I\}\_\{HE\}, which contains less redundant intra\-class variation\. Fig\.[10](https://arxiv.org/html/2607.02637#A1.F10)shows CIFAR\-10 examples, and Tab\.[7](https://arxiv.org/html/2607.02637#A1.T7)shows thatℐH​O\\mathcal\{I\}\_\{HO\}has higher intra\-class similarity thanℐH​E\\mathcal\{I\}\_\{HE\}across datasets\.

Table 7:Average intra\-class similarity ofℐH​O\\mathcal\{I\}\_\{HO\}andℐH​E\\mathcal\{I\}\_\{HE\}across datasets\.ℐH​O\\mathcal\{I\}\_\{HO\}consistently gets higher intra\-class similarity\.
## Appendix BMotivation and Theoretical Justification of\\ho\-\\hePartition

In Sec\.[2](https://arxiv.org/html/2607.02637#S3.F2), we argue that the proposed\\hohepartition admits a theoretical justification under a graph\-based view\. In this section, we further elaborate the motivation behind the\\hoand\\hedesign and provide a formal analysis showing why optimizing with respect to this partition is well\-aligned with the learning objective in our setting\.

### B\.1Semantic\-mode diversity and Variation Diversity

To clarify the design of our real data partition, we first distinguish between semantic\-mode diversity and variation diversity\. We provide a PCA visualization in Figure[11](https://arxiv.org/html/2607.02637#A2.F11)\. In Figure[11\(a\)](https://arxiv.org/html/2607.02637#A2.F11.sf1), the real data are partitioned into two sets based on the distance of each instance to the class centroid: the “near” set contains canonical patterns, while the “far” set consists of patterns that are less similar to the semantic core and therefore exhibit higher semantic\-mode diversity\. Each of them cover the subregion of original feature space\. In contrast, Figure[11\(b\)](https://arxiv.org/html/2607.02637#A2.F11.sf2)illustrates our\\ho/\\hepartition, where both\\hoand\\hecover all semantic modes present in the real data, and the diversity in\\hemanifests as variation diversity, rich variations around each semantic mode across the entire dataset\.

Thus, Figure[11\(a\)](https://arxiv.org/html/2607.02637#A2.F11.sf1)illustrates semantic\-mode diversity: the “near” and “far” sets are distinguished by the presence of different semantic modes in each subregion\. In contrast, Figure[11\(b\)](https://arxiv.org/html/2607.02637#A2.F11.sf2)shows the\\ho/\\hepartition, where the two sets are distinguished by the variation level within each mode across the entire feature space, rather than by different modes themselves\.

### B\.2Why we choose\\ho\-\\hePARTITION

Standard image augmentations improve model performance by increasing variation within the dataset while preserving the main semantics\. Inspired by this, we hypothesize that, when curating synthetic data, if we can construct a set that covers all semantic modes while exhibiting high variation within each mode, we can similarly expect performance gains\. Based on this idea, we propose the\\ho/\\hepartition of the real data, which preserves all modes \(as shown in Figure[11\(b\)](https://arxiv.org/html/2607.02637#A2.F11.sf2)\) but explicitly distinguishes them by their level of intra\-mode variation, and use them as guidance for synthetic data selection\.

### B\.3Nearest\-neighbor representative coverage

For each real featurerir\_\{i\}, define its nearest neighbor by

j​\(i\)=arg⁡minj≠i⁡d​\(ri,rj\)\.j\(i\)=\\arg\\min\_\{j\\neq i\}d\(r\_\{i\},r\_\{j\}\)\.The\\hoset is the image of this nearest\-neighbor map,H=\{rj​\(i\):i∈\[n\]\}H=\\\{r\_\{j\(i\)\}:i\\in\[n\]\\\}, and the\\heset isE=R∖HE=R\\setminus H\. Equivalently, if we build the directed 1\-nearest\-neighbor graph by drawing an edgeri→rj​\(i\)r\_\{i\}\\to r\_\{j\(i\)\}, thenHHis the set of nodes with positive in\-degree andEEis the set of zero in\-degree nodes\.

###### Proof of Proposition[1](https://arxiv.org/html/2607.02637#Thmproposition1)\.

For anyrir\_\{i\}, the featurerj​\(i\)r\_\{j\(i\)\}belongs toHHby construction andj​\(i\)≠ij\(i\)\\neq i\. Therefore

minh∈H,h≠ri⁡d​\(ri,h\)≤d​\(ri,rj​\(i\)\)=minr∈R,r≠ri⁡d​\(ri,r\)\.\\min\_\{h\\in H,\\,h\\neq r\_\{i\}\}d\(r\_\{i\},h\)\\leq d\(r\_\{i\},r\_\{j\(i\)\}\)=\\min\_\{r\\in R,\\,r\\neq r\_\{i\}\}d\(r\_\{i\},r\)\.SinceH⊆RH\\subseteq R, the reverse inequality holds automatically, giving equality forHH\.

Conversely, letA⊆RA\\subseteq Rsatisfy the equality in Proposition[1](https://arxiv.org/html/2607.02637#Thmproposition1)for everyrir\_\{i\}\. Under the unique\-nearest\-neighbor assumption, the only element ofR∖\{ri\}R\\setminus\\\{r\_\{i\}\\\}that attains the right\-hand side isrj​\(i\)r\_\{j\(i\)\}\. Hencerj​\(i\)∈Ar\_\{j\(i\)\}\\in Afor allii, and thusH⊆AH\\subseteq A\. SinceHHitself satisfies the equality, it is the unique inclusion\-minimal subset with this property\.

Finally, setε=maxi⁡d​\(ri,rj​\(i\)\)\\varepsilon=\\max\_\{i\}d\(r\_\{i\},r\_\{j\(i\)\}\)\. The equality just proved implies

minh∈H,h≠ri⁡d​\(ri,h\)=d​\(ri,rj​\(i\)\)≤ε\\min\_\{h\\in H,\\,h\\neq r\_\{i\}\}d\(r\_\{i\},h\)=d\(r\_\{i\},r\_\{j\(i\)\}\)\\leq\\varepsilonfor everyri∈Rr\_\{i\}\\in R, yielding the stated coverage radius\. ∎

This result gives the precise sense in which\\hocontains canonical representatives: it is the smallest set that preserves every training point’s nearest\-neighbor reconstruction cost\. The complement\\heis therefore not “noise” or “outliers” by definition; rather, it contains real samples that are not needed as nearest\-neighbor representatives for other samples and thus encode less redundant local variation\.

### B\.4Geometric interpretation of the diversity score\.

The diversity score has a simple angular meaning\. For an\\heanchorrr, letar=h​\(r\)−ra\_\{r\}=h\(r\)\-rbe the direction fromrrto its nearest\\honeighbor\. If a synthetic candidatezzmoves fromrrinside a cone of half\-angleθ\\thetaaroundara\_\{r\}, then

⟨z−r‖z−r‖,ar‖ar‖⟩≥cos⁡θ⇒Sdiv\\he​\(z,r\)≤−cos⁡θ\.\\left\\langle\\frac\{z\-r\}\{\\\|z\-r\\\|\},\\frac\{a\_\{r\}\}\{\\\|a\_\{r\}\\\|\}\\right\\rangle\\geq\\cos\\theta\\qquad\\Rightarrow\\qquad S^\{\\he\}\_\{\\mathrm\{div\}\}\(z,r\)\\leq\-\\cos\\theta\.Thus increasingα\\alphasuppresses candidates that are close to an\\heanchor only by drifting back toward the canonical\\hodirection\. This is the intended complement to the fidelity term: fidelity keeps the selected sample semantically aligned with an anchor, while diversity discourages repeated canonical collapse\.

### B\.5Graph interpretation of the\\hohepartition\.

In this section, we discuss the properties of the\\hohepartition and prove its optimality in our setting\. As defined\\hohein Eqs\.[2](https://arxiv.org/html/2607.02637#S3.E2),, we can review such partitions in the graph setting\. In the original dataset, each feature of the image could be viewed as the node, and the cosine similarity between different features could be used as the distance connecting each pair of nodes \(higher similarity corresponds to a smaller distance and less cost for transfer\)\. Then we can compose a fully connected graphG=\(V,E\)G=\(V,E\), andVVis the set of nodes, and\|V\|=n\|V\|=n, andEEis the set of edges,\|E\|=n×n\|E\|=n\\times n\.

To find the “hubs" in the graph, we traverse all nodes in the graph and connect each node to its single nearest neighbor, forming a directed graphG′=\(V,E′\)G^\{\\prime\}=\(V,E^\{\\prime\}\), where\|E′\|=n\|E^\{\\prime\}\|=n\. Under this setting,\\hois equivalent to the collection of nodes whose in\-degree is positive\.

So in the graph setting, the definition of\\hocould be rewritten as:

HO=\{a∈V\|∃b∈V∖\{a\},such that​∀c∈V∖\{a,b\},d​\(a,b\)≤d​\(c,b\)\}\\mathrm\{HO\}=\\left\\\{a\\in V\\ \\middle\|\\ \\exists b\\in V\\setminus\\\{a\\\},\\text\{ such that \}\\forall c\\in V\\setminus\\\{a,b\\\},d\(a,b\)\\leq d\(c,b\)\\right\\\}
Based on the definition above, we can derive the following properties:

1. 1\.To reach any nodebbin the graph, starting from a node in\\ho\(different frombb\) consistently yields the minimal cost\. Such property implies that learning the pattern in\\ho, and then we can reconstruct the whole original feature space with the smallest cost\.
2. 2\.Suppose, in the original graph, each node has a single nearest neighbor\. Then, theoretically,\\hois a minimal subset of nodes such that every node in the graph can be reached from some node in\\howith minimal cost\.

![Refer to caption](https://arxiv.org/html/2607.02637v1/x14.png)\(a\)Centroid Partition
![Refer to caption](https://arxiv.org/html/2607.02637v1/x15.png)\(b\)\\ho\-\\hePartition

Figure 11:PCA visualization in ImageNet\-1K \(n03769881\), using different partition strategiesTraining downstream models on a given dataset can be interpreted as optimizing the model to capture the full set of patterns in the underlying feature space\. From a graph perspective, once the model learns the patterns in\\ho, it can often propagate this knowledge to other modes with relatively little additional cost\. As a result, after sufficient training, downstream models naturally exhibit a preference for\\ho, where the patterns are easier to learn and generalize\.

Therefore, motivated by this insight, we propose a synthetic data selection strategy built on the\\hohepartition to better unlock the value of synthetic samples for downstream training\.

## Appendix CGenerator and Discriminator Biases in the\\ho/\\heSplit

Echoing the discussion ofDownstream Model Preferencein Sec\.[3\.1](https://arxiv.org/html/2607.02637#S3.SS1),ℐH​O\\mathcal\{I\}\_\{HO\}andℐH​E\\mathcal\{I\}\_\{HE\}together constitute a complete real dataset, and when such data are used to train downstream models, these characteristics may be implicitly inherited\.

![Refer to caption](https://arxiv.org/html/2607.02637v1/x16.png)Figure 12:The cosine similarity measures how alike instances are within the individual class in CIFAR\-10\. From the figure, the mean cosine similarity of each class in the synthetic dataset follows the trend of the real dataset, indicating that the generative model effectively captures patterns from real data\. However, the synthetic data consistently exhibit higher mean cosine similarity than the real data, suggesting that within each class, synthetic instances are more homogeneous and share greater similarity with each other compared to real instances\.This figure release that synthetic dataset have limitation of diversity in each individual class\.For Generative Models, we investigate the relationship between generated instances and real instances inℐH​O\\mathcal\{I\}\_\{HO\}andℐH​E\\mathcal\{I\}\_\{HE\}\. As shown in Figure[4](https://arxiv.org/html/2607.02637#S3.F4), samples generated by EDM\[[27](https://arxiv.org/html/2607.02637#bib.bib42)\]exhibit higher similarity to those in theℐH​O\\mathcal\{I\}\_\{HO\}, highlighting the well\-known limitation of reduced diversity in generative data\. SinceℐH​O\\mathcal\{I\}\_\{HO\}samples capture canonical patterns that are easier for generative models to learn, this bias towardℐH​O\\mathcal\{I\}\_\{HO\}leads to higher intra\-class similarity compared to real datasets, thereby reducing overall diversity, as shown in Figure[12](https://arxiv.org/html/2607.02637#A3.F12)\.

For Discriminators, trained on real data, we observe a clear performance gap betweenℐH​O\\mathcal\{I\}\_\{HO\}andℐH​E\\mathcal\{I\}\_\{HE\}, even when evaluation is conducted on the original training set\. As shown in Figure[13](https://arxiv.org/html/2607.02637#A3.F13), we run inference using off\-the\-shelf pretrained models \(referring Table[8](https://arxiv.org/html/2607.02637#A3.T8)\) on the training datasets and consistently find higher accuracy onℐH​O\\mathcal\{I\}\_\{HO\}than onℐH​E\\mathcal\{I\}\_\{HE\}\. We attribute this gap to differences in canonicality:ℐH​O\\mathcal\{I\}\_\{HO\}contains more canonical instances that are easier for models to fit, whereasℐH​E\\mathcal\{I\}\_\{HE\}demands modeling a broader range of intra\-class variations\.

![Refer to caption](https://arxiv.org/html/2607.02637v1/x17.png)Figure 13:Performance gap betweenℐH​O\\mathcal\{I\}\_\{HO\}andℐH​E\\mathcal\{I\}\_\{HE\}across various datasets and pretrained models\. ForSVHN,CIFAR\-10,Tiny\-ImageNet, andImageNet\-1K, we evaluate multiple ResNet and ViT variants\. Descriptions and links for all pretrained models are provided in Table[8](https://arxiv.org/html/2607.02637#A3.T8)\. The experimental results reveal a striking fact: even within the training set, a clear performance gap persists betweenℐH​O\\mathcal\{I\}\_\{HO\}andℐH​E\\mathcal\{I\}\_\{HE\}\.Based on this observation,*training discriminators with generative data can be risky, as it may amplify the limitations already present in those trained with real data, further biasing the model toward canonical samples and degrading performance on real data inference*\.

Table 8:Pretrained discriminators used in our experiments\.DatasetModelLinkSVHNResNet\-18[https://huggingface\.co/edadaltocg/resnet18\_svhn](https://huggingface.co/edadaltocg/resnet18_svhn)ResNet\-50[https://huggingface\.co/edadaltocg/resnet50\_svhn](https://huggingface.co/edadaltocg/resnet50_svhn)ViT\-Base[https://huggingface\.co/edadaltocg/vit\_base\_patch16\_224\_in21k\_ft\_svhn](https://huggingface.co/edadaltocg/vit_base_patch16_224_in21k_ft_svhn)CIFAR\-10ResNet\-18[https://huggingface\.co/SamAdamDay/resnet18\_cifar10](https://huggingface.co/SamAdamDay/resnet18_cifar10)ResNet\-50[https://huggingface\.co/anonauthors/cifar10\-timm\-resnet50](https://huggingface.co/anonauthors/cifar10-timm-resnet50)ViT\-Base[https://huggingface\.co/nateraw/vit\-base\-patch16\-224\-cifar10](https://huggingface.co/nateraw/vit-base-patch16-224-cifar10)Tiny\-ImageNetResNet\-18[https://github\.com/zeyuanyin/tiny\-imagenet](https://github.com/zeyuanyin/tiny-imagenet)ResNet\-50[https://github\.com/zeyuanyin/tiny\-imagenet](https://github.com/zeyuanyin/tiny-imagenet)ViT\-L[https://github\.com/ehuynh1106/TinyImageNet\-Transformers](https://github.com/ehuynh1106/TinyImageNet-Transformers)ImageNet\-1KAll modelstimm\[[66](https://arxiv.org/html/2607.02637#bib.bib68)\]
## Appendix DSynthetic Data Selection Details

Following the previous discussion on\\hoand\\he, this section introduces our synthetic data selection strategy designed to mitigate the limitations of training discriminators on generative data\. The key difference from prior approaches is the incorporation of a diversity score\. During selection, in addition to the fidelity score, which measures how closely synthetic samples resemble real data, the diversity score quantifies how far a synthetic instance departs from canonical patterns\. As illustrated in Figure[14](https://arxiv.org/html/2607.02637#A4.F14), when referencing real instances, a synthetic sample receives a higher diversity score if its direction from the\\heset deviates more strongly from the direction toward the canonical pattern\.

![Refer to caption](https://arxiv.org/html/2607.02637v1/x18.png)Figure 14:Examples of\\heinstances and their most similar counterparts in\\ho\(based on MoCo V3 features\) from ImageNet\-1K\.Red arrowsdenote vectors formed by each image pair\. To capture the diversity within\\he, our algorithm prioritizes synthetic images that, when paired with a\\heinstance, yield vectors that deviate from the red\-arrow directions\.
## Appendix EExperiment Details and Full Results

### E\.1Generator usage in the main paper

To validate our strategy, we train models on synthetic datasets and evaluate them on real data across two settings: in\-domain and out\-of\-domain testing\. In these experiments, we utilize publicly available generative models and synthetic dataset, and conduct training using their released code repositories\. The corrsponding responses are illustrated in Table[9](https://arxiv.org/html/2607.02637#A5.T9)\. For the generator usage in Sec\.[4\.4](https://arxiv.org/html/2607.02637#S4.SS4), we follow the original papers and use the generators reported in\[[62](https://arxiv.org/html/2607.02637#bib.bib27),[3](https://arxiv.org/html/2607.02637#bib.bib26)\]

Table 9:Public repositories and resources used in this work\. The first three rows correspond to classifier training repositories, and the last two rows correspond to generative models\.
### E\.2IN\-1K training recipe

We use the timm codebase to train models for the IN\-1K setting\. Since training in IN\-1K involves a relatively complex configuration and the final performance can vary with different settings, we report the exact configuration used in the main paper to facilitate reproduction of our empirical results\. We trainvit\_base\_patch16\_224andresnet50from scratch, and fine\-tunevit\_base\_patch16\_224\.augreg\_in21kandresnetv2\_50x1\_bit\.goog\_in21kpretrained on ImageNet\-21K\. The concrete settings are shown in Figure[15](https://arxiv.org/html/2607.02637#A5.F15)\.

Listing 1:ViT\-B/16 training from scratch\.\./distributed\_train\.sh"$NUM\_GPU"\\

\-\-data\-dir"$IMAGENET\_DIR"\\

\-\-modelvit\_base\_patch16\_224\\

\-\-epochs120\-\-batch\-size256\\

\-\-optadamw\-\-weight\-decay0\.05\\

\-\-schedcosine\-\-warmup\-epochs5\\

\-\-lr\-base5e\-4\-\-lr\-base\-size1024\\

\-\-aarand\-m9\-mstd0\.5\-inc1\\

\-\-mixup0\.8\-\-cutmix1\.0\-\-smoothing0\.1\\

\-\-drop\-path0\.1\-\-channels\-last\-\-amp

Listing 2:ResNet\-50 training from scratch\.\./distributed\_train\.sh"$NUM\_GPU"\\

\-\-data\-dir"$IMAGENET\_DIR"\\

\-\-modelresnet50\\

\-\-epochs100\-\-batch\-size64\\

\-\-optsgd\-\-momentum0\.9\-\-weight\-decay1e\-4\\

\-\-schedcosine\-\-warmup\-epochs3\-\-cooldown\-epochs10\\

\-\-lr0\.025\-\-smoothing0\.1\\

\-\-mixup0\.2\-\-cutmix1\.0\\

\-\-aarand\-m9\-mstd0\.5\-inc1\\

\-\-drop\-path0\.05\-\-amp\-\-channels\-last

Listing 3:Fine\-tuning ViT\-B/16 \(IN\-21K init\)\.\./distributed\_train\.sh"$NUM\_GPU"\\

\-\-data\-dir"$IMAGENET\_DIR"\\

\-\-modelvit\_base\_patch16\_224\.augreg\_in21k\\

\-\-pretrained\\

\-\-epochs50\-\-batch\-size512\\

\-\-optadamw\-\-weight\-decay0\.05\\

\-\-schedcosine\-\-warmup\-epochs5\\

\-\-lr\-base5e\-5\-\-lr\-base\-size1024\\

\-\-aarand\-m9\-mstd0\.5\-inc1\\

\-\-mixup0\.2\-\-cutmix0\.8\-\-smoothing0\.1\\

\-\-drop\-path0\.1\-\-channels\-last\-\-amp

Listing 4:Fine\-tuning ResNet\-50 \(IN\-21K init\)\.\./distributed\_train\.sh"$NUM\_GPU"\\

\-\-data\-dir"$IMAGENET\_DIR"\\

\-\-modelresnetv2\_50x1\_bit\.goog\_in21k\\

\-\-pretrained\\

\-\-epochs50\-\-batch\-size64\\

\-\-optsgd\-\-momentum0\.9\-\-weight\-decay1e\-4\\

\-\-schedcosine\-\-warmup\-epochs3\-\-cooldown\-epochs10\\

\-\-lr0\.025\-\-smoothing0\.1\\

\-\-mixup0\.2\-\-cutmix1\.0\\

\-\-aarand\-m9\-mstd0\.5\-inc1\\

\-\-drop\-path0\.05\-\-amp\-\-channels\-last

Figure 15:Training recipes for different backbones\. Top row: from\-scratch training\. Bottom row: fine\-tuning with ImageNet\-21K initialization\.
### E\.3In\-domain classification accuracy

Echoing Sec\.[4\.2](https://arxiv.org/html/2607.02637#S4.SS2), we evaluate model performance on SVHN, CIFAR\-10, and Tiny\-ImageNet to enable hierarchical assessment across datasets of varying complexity, and further conduct experiments on ImageNet\-1K to simulate the large\-scale setting\.

SVHN:We compare our method with prior approaches, and the results are reported in Table[10](https://arxiv.org/html/2607.02637#A5.T10)\. When trained exclusively on synthetic datasets, model performance can surpass that of models trained on real data as the training volume increases\. Among different strategies, ours consistently achieves the best performance across varying training data scales\.

Table 10:ResNet18performance on theSVHNdataset across different training sizes\. As the amount of synthetic training data increases, model performance improves\. Thegreen bold textindicates the highest accuracy across different data selection strategies\.CIFAR\-10:We compare our method with prior approaches, and the results are reported in Table[11](https://arxiv.org/html/2607.02637#A5.T11)\. When trained exclusively on synthetic datasets, model performance can surpass that of models trained on real data as the training volume increases\. Among different strategies, ours consistently achieves the best performance across varying training data scales\.

Table 11:ResNet18performance on theCIFAR\-10dataset across different training sizes\. As the amount of synthetic training data increases, model performance improves\. Thegreen bold textindicates the highest accuracy across different data selection strategies\.Tiny\-ImageNet:We compare our method with prior approaches, and the results are reported in Table[12](https://arxiv.org/html/2607.02637#A5.T12)\. In this more complex setting, we first train models exclusively on synthetic data, where the performance gap between synthetic\- and real\-trained models is around 10%\. To effectively validate the utility of synthetic data in discriminator training, we use synthetic data as an augmentation to real data\. Under this setting, our method consistently outperforms alternatives across different experimental configurations\.

Table 12:Performance of EfficientNet\-B0 and Resnet50 on theTiny\-ImageNetdataset with different synthetic dataset\. Results are reported as Mean±\\pmStd\. \(relative difference from the real\-data baseline\)\. The best results in each column are highlighted\. As the amount of synthetic training data increases, model performance improves\. The experiment results demonstrate the importance of the trade\-off between fidelity and diversity\. The original data size is 100K\.
### E\.4Out\-of\-domain classification accuracy

Assessing models’ robustness, OOD evaluations are implemented \( Sec\.[4\.3](https://arxiv.org/html/2607.02637#S4.SS3)in main paper\)\. We report the concrete performance values in each setting as blow:

SVHN:We evaluate OOD performance using the extra split of SVHN\[[42](https://arxiv.org/html/2607.02637#bib.bib12)\]and test subset of distorted SVHN\. Detailed results are presented in Table[13](https://arxiv.org/html/2607.02637#A5.T13)\.

Table 13:OODPerformance comparison onSVHN\. The best performance for each dataset is highlighted inbold\.CIFAR\-10:We use CIFAR\-10\-Warehouse\[[52](https://arxiv.org/html/2607.02637#bib.bib76)\]as a benchmark, the subset of data in this benchmark is illustrated in Figure[16](https://arxiv.org/html/2607.02637#A5.F16)\. The inference results are presented in Table[14](https://arxiv.org/html/2607.02637#A5.T14)\.

![Refer to caption](https://arxiv.org/html/2607.02637v1/x19.png)![Refer to caption](https://arxiv.org/html/2607.02637v1/x20.png)![Refer to caption](https://arxiv.org/html/2607.02637v1/x21.png)![Refer to caption](https://arxiv.org/html/2607.02637v1/x22.png)![Refer to caption](https://arxiv.org/html/2607.02637v1/x23.png)![Refer to caption](https://arxiv.org/html/2607.02637v1/x24.png)

Figure 16:Subsets of CIFAR\-10\-Warehouse dataset for CIFAR\-10 OOD testing\.![Refer to caption](https://arxiv.org/html/2607.02637v1/imgs/vis.png)Figure 17:Subsets of IN\-1K OOD dataTable 14:OODPerformance comparison onCIFAR\-10\. The best performance for each dataset is highlighted inbold\.Tiny\-ImageNet:We utilize Tiny\-ImageNet\-C\[[25](https://arxiv.org/html/2607.02637#bib.bib11)\]for the evaluation of OOD, which incorporates various types of corruption\. We classify them into three types: color\-variation set \(\\iebrightness adjustment, contrast variation\), noise\-variation set \(\\iepixelation, Gaussian noise, motion blur\), and compression\-variation set \(\\ieJPEG compression\)\. The results are presented in Table[15](https://arxiv.org/html/2607.02637#A5.T15)\.

Table 15:OODPerformance comparison onTiny\-ImageNet\. The best performance for each dataset is highlighted inbold\.ImageNet\-1K:We use five ImageNet\-1K OOD, illustrated in Figure[17](https://arxiv.org/html/2607.02637#A5.F17):

1\) ImageNet\-V2\[[47](https://arxiv.org/html/2607.02637#bib.bib93)\]is constructed to closely match the distribution of the original ImageNet\-1K, containing 50,000 images across the same 1,000 classes as the original validation set\.

2\) ImageNet\-Sketch\[[61](https://arxiv.org/html/2607.02637#bib.bib99)\]consists of black\-and\-white sketches covering all ImageNet\-1K classes, with 50 images per class\.

3\) ImageNet\-C\[[26](https://arxiv.org/html/2607.02637#bib.bib98)\]evaluates model robustness to common corruptions such as noise, blur, weather effects, and digital distortions by applying perturbations with severity levels from 1 to 5 to the original ImageNet validation images\.

4\) ImageNet\-Drawing\[[48](https://arxiv.org/html/2607.02637#bib.bib95)\]is derived from the ImageNet validation set, where images are transformed into drawing styles using generative adversarial networks and image processing techniques\.

5\) ImageNet\-Cartoon\[[48](https://arxiv.org/html/2607.02637#bib.bib95)\]is also derived from the ImageNet validation set, where images are transformed into cartoon styles\.

We categorize them into two main groups based on the sources of image\. 1\) Original OOD: ImageNet\-V2 and ImageNet\-Sketch; 2\) Derivative OOD: ImageNet\-C, ImageNet\-Drawing and ImageNet\-Cartoon\. The inference results is illustrated in the Table[16](https://arxiv.org/html/2607.02637#A5.T16),[17](https://arxiv.org/html/2607.02637#A5.T17)\.

Table 16:OODPerformance comparison onImageNet\-1Kfor ViT\-B/16\. The best performance for each dataset is highlighted inbold\.Table 17:OODPerformance comparison onImageNet\-1Kfor ResNet50\. The best performance for each dataset is highlighted inbold\.Table 18:Experimental configurations for different datasets, including generator, classifier, and training dataset size\.

## Appendix FAdditional Analysis

### F\.1Limitation of CLIP filter

Reviewing the experimental results, we observe that using CLIP as a filter to select synthetic data consistently leads to the worst performance when training discriminators\. Upon inspecting the selected samples, we attribute this degradation to the dominance of monotonous instances\. As illustrated in Figure[18](https://arxiv.org/html/2607.02637#A6.F18), high\- and low\-CLIP\-score examples in Tiny\-ImageNet generated by EDM highlight this issue, while Figure[19](https://arxiv.org/html/2607.02637#A6.F19)presents corresponding cases in ImageNet\-1K generated by EDM2\. When constructing datasets from a synthetic pool, prioritizing high\-CLIP\-score instances produces collections enriched with canonical patterns but lacking diversity, ultimately resulting in weaker performance on both in\-domain and out\-of\-domain evaluations\.

![Refer to caption](https://arxiv.org/html/2607.02637v1/x25.png)Figure 18:Instances with high and low CLIP\-Scores in EDM generated Tiny\-ImageNet\.![Refer to caption](https://arxiv.org/html/2607.02637v1/x26.png)Figure 19:Instances with high and low CLIP\-Scores in EDM2 generated ImageNet\-1K\.
### F\.2Feature Extractor Ablation

Our strategy is built upon image feature representations; therefore, we further investigate whether the choice of feature extractor influences the final performance of classifiers\. In this ablation study, we conduct experiments on two datasets: CIFAR\-10 and ImageNet\-100\. The experimental configurations are summarized in Table[18](https://arxiv.org/html/2607.02637#A5.T18)\. Specifically, for CIFAR\-10, we use EDM to generate 1M synthetic samples and apply our strategy to select 100K training instances, which are then used to train a ResNet\-18 model\.

## Appendix GImplement Details

### G\.1Code: scoring

Listing 5:Synthetic alignment scoringdefcompute\_synthetic\_common\_alignment\_scores\(

syn\_normed,\#\[N,D\]normalizedsyntheticfeatures

real\_common\_normed\#\[M\_1,D\]normalizedreal\-common

synthetic\_common\_fidelity=syn\_normed@real\_common\_normed\.T

\#diversityscore

centroid\_common=F\.normalize\(real\_common\_normed\.mean\(dim=0,keepdim=True\),dim=1\)

A=syn\_normed\.unsqueeze\(1\)\-real\_common\_normed\.unsqueeze\(0\)\#\[N,M\_1,D\]

B=centroid\_common\-real\_common\_normed\#\[M\_1,D\]

numerator=\(A\*B\.unsqueeze\(0\)\)\.sum\(dim=\-1\)\#\[N,M\_1\]

A\_norm=A\.norm\(dim=\-1\)\#\[N,M\_1\]

B\_norm=B\.norm\(dim=\-1\)\.unsqueeze\(0\)\#\[1,M\_1\]

synthetic\_common\_diversity=numerator/\(A\_norm\*B\_norm\+1e\-8\)\#\[N,M\_1\]

synthetic\_common\_score=synthetic\_common\_fidelity\-synthetic\_common\_diversity\#\[N,M\_1\]

returnsynthetic\_common\_fidelity,synthetic\_common\_diversity,synthetic\_common\_score

defcompute\_synthetic\_rare\_alignment\_scores\(

syn\_normed=None,real\_rare\_normed=None,

real\_np=None,\#realfeaturesmatrix

class\_rare\_2\_common\_in\_real\_dict=None\):

synthetic\_rare\_fidelity=syn\_normed@real\_rare\_normed\.T\#\[N,M\_2\]

rare\_2\_common\_in\_real\_dict=class\_rare\_2\_common\_in\_real\_dict

common\_counterpart\_index=\[i\["common\_image"\]foriinrare\_2\_common\_in\_real\_dict\.values\(\)\]

common\_counterpart\_matrix=get\_real\_subset\_features\_matrix\(real\_np,common\_counterpart\_index\)\#\[M\_2,D\]

normalized\_common\_counterpart\_matrix=F\.normalize\(common\_counterpart\_matrix,dim=1\)\#\[M\_2,D\]

rare\_2\_common\_matrix=normalized\_common\_counterpart\_matrix\-real\_rare\_normed\#shape:\[M2,D\]

norm\_rare\_2\_common\_matrix=rare\_2\_common\_matrix/rare\_2\_common\_matrix\.norm\(dim=\-1,keepdim=True\)\#Normalizetounitvectors

rare\_2\_synthetic\_matrix=syn\_normed\.unsqueeze\(1\)\-real\_rare\_normed\.unsqueeze\(0\)\#\[N,M2,D\]

norm\_rare\_2\_synthetic\_matrix=rare\_2\_synthetic\_matrix/rare\_2\_synthetic\_matrix\.norm\(dim=\-1,keepdim=True\)\#\[N,M2,D\]

cos\_sim\_matrix=\(norm\_rare\_2\_synthetic\_matrix\*norm\_rare\_2\_common\_matrix\.unsqueeze\(0\)\)\.sum\(dim=\-1\)

synthetic\_rare\_diversity=cos\_sim\_matrix\#\[N,M2\]

synthetic\_rare\_score=synthetic\_rare\_fidelity\-synthetic\_rare\_diversity\#\[N,M2\]

returnsynthetic\_rare\_fidelity,synthetic\_rare\_diversity,synthetic\_rare\_score

### G\.2Code: selection

Listing 6:Top\-k synthetic selectiondefget\_the\_highest\_score\_index\(matrix,synthetic\_name\_list,top\_k=250,top\_n=2,reverse=False\):

ifmatrix\.shape\[0\]\>matrix\.shape\[1\]:

matrix=matrix\.T

number\_of\_real=matrix\.shape\[0\]

number\_of\_synthetic=matrix\.shape\[1\]

ifreverse:

matrix=\-matrix

\#foreachrowreturnthetop\-nlargestcolumnindicesandvalues

topk\_values,topk\_indices=torch\.topk\(matrix,top\_n,dim=1\)\#shape:\(rows,top\_n\)

\#print\("theshapeofmatrix:",matrix\.shape\)

assertlen\(synthetic\_name\_list\)==matrix\.shape\[1\],\\

"Syntheticimagenameslengthmustmatchthenumberofsyntheticfeatures\."

retrieval\_syn\_image\_and\_scores=\{\}

\#iteratethrougheachrow

forindices,valuesinzip\(topk\_indices,topk\_values\):

foridx,valinzip\(indices,values\):

current\_image\_name=synthetic\_name\_list\[idx\.item\(\)\]

current\_value=retrieval\_syn\_image\_and\_scores\.get\(current\_image\_name,\-9999\)

ifval\.item\(\)\>current\_value:

retrieval\_syn\_image\_and\_scores\[current\_image\_name\]=val\.item\(\)

\#checkuniquecount

assertlen\(set\(retrieval\_syn\_image\_and\_scores\.keys\(\)\)\)\>=top\_k,\\

f"Thenumberofuniquetop\-\{top\_n\}columnindicesislessthantop\_k\."

returnlist\(retrieval\_syn\_image\_and\_scores\.keys\(\)\),retrieval\_syn\_image\_and\_scores

## Appendix HLimitations

This work has several limitations, which also point to promising directions for future research\.

1\) Dependence on generator quality\.Our method operates as a post\-generation curation strategy built on top of off\-the\-shelf generators, without intervening in or steering the generation process itself\. As a result, the upper bound of synthetic data utility is inherently constrained by the quality of the generator\. When the generator is too weak to produce realistic images, curation may largely reduce to selecting samples with relatively higher fidelity, while diversity\-aware selection may become less beneficial because low\-fidelity samples can further weaken downstream performance\. Nevertheless, as modern generative models continue to improve, synthetic data is becoming increasingly realistic, making post\-generation curation more meaningful for unlocking its downstream utility\.

2\) Limited to unimodal image settings\.Our current method focuses on curating synthetic image data and does not extend to multimodal settings\. As vision\-language models continue to advance, synthetic datasets that support multimodal training are becoming increasingly important\. However, multimodal curation involves data formats beyond images and requires modeling interactions across modalities, which are not addressed by the current framework\. Extending post\-generation curation to multimodal data is therefore an important and meaningful direction for future work\.

3\) Reliance on real reference data\.Our method requires a real reference set to construct theℐH​O\\mathcal\{I\}\_\{HO\}/ℐH​E\\mathcal\{I\}\_\{HE\}partition, which is standard in reference\-based curation\. In practice, however, only a small reference set is often sufficient, since the partition depends more on relative intra\-class similarity than on absolute data scale\.

4\) Dependence on pretrained feature extractors\.Our scoring procedure relies on pretrained feature extractors\. Although Fig\. 9 shows that the method is only modestly sensitive to the choice of encoder, a severely domain\-mismatched feature extractor may still degrade partition quality\. In practice, we recommend using strong general\-purpose encoders, such as SigLIP, when domain\-specific alternatives are unavailable\.

## Appendix IBroader impacts

Our work on synthetic data selection raises several ethical considerations that warrant discussion\.Bias Amplification: Our HOMO\-HETERO partitioning strategy could potentially amplify biases present in the original data if the HOMO set disproportionately represents certain demographic groups or patterns\. We mitigate this by explicitly incorporating diversity metrics in our selection criteria\.Environmental Impact: While synthetic data generation incurs computational costs, our selective curation approach reduces the environmental footprint by enabling comparable performance with smaller datasets, requiring fewer training iterations and less storage\.Privacy Preservation: Although synthetic data generally poses fewer privacy risks than real data, our method does not specifically filter for instances that might inadvertently resemble real individuals\. Practitioners should implement additional privacy checks when deploying our method in sensitive domains\.Responsible Use: We acknowledge that synthetic data technology could be misused for creating deceptive content\. Our method is designed for legitimate research and educational purposes, specifically to improve model training efficiency while maintaining performance standards\. We encourage adherence to established guidelines for synthetic data use and advocate for continued development of detection methods to distinguish synthetic from real content\.