Hierarchical Data Selection via Manifold Coverage and Sparse Feature Coverage in LLM Post-training

# Hierarchical Data Selection via Manifold Coverage and Sparse Feature Coverage in LLM Post-training
Source: [https://arxiv.org/html/2608.16927](https://arxiv.org/html/2608.16927)
Written by AAAI Press Staff1 AAAI Style Contributions by Peter Patel Schneider, Sunil Issar, J\. Scott Penberthy, George Ferguson, Hans Guesgen, Francisco Cruz\\equalcontrib\\corresponding, Marc Pujol\-Gonzalez\\equalcontrib\\correspondingPeng Sun1, Yi Yang1, Antong Zhang2, Chunxiao Li3, Yanbo Wang4, Dianbo Liu5, Xin Chen6, Kai Yu7, Lu Chen7, Tianfan Fu1\\corresponding

###### Abstract

As supervised fine\-tuning data continues to scale, selecting high\-value subsets from large candidate pools is crucial for reducing training cost and improving model performance\. Existing methods often measure diversity directly in the original embedding space, where geometric metrics entangle dominant semantic directions, fine\-grained supervision differences, and local noise\. We address this limitation by formulating data selection as a coarse\-to\-fine hierarchical coverage problem and propose MASS\. MASS learns low\-dimensional principal manifold coordinates with a dense autoencoder for coarse semantic grouping, and then performs quality\-aware sparse feature coverage within each group using a TopK sparse autoencoder\. Experiments on Vision Flan and LLaVA\-CoT show that MASS consistently outperforms strong data selection baselines across multiple budgets, and in several settings matches or surpasses full data training with only a small subset of data\.

## 1Introduction

As the scale of supervised fine\-tuning data for large models continues to grow, selecting high\-value subsets from large candidate pools has become an important problem for reducing training cost and improving model performance\(Ivisonet al\.[2025](https://arxiv.org/html/2608.16927#bib.bib25); Liuet al\.[2025](https://arxiv.org/html/2608.16927#bib.bib26)\)\. Existing data selection methods often use embeddings to characterize data distributions, and maintain sample diversity and coverage through similarity measurement and clustering\(Debet al\.[2025](https://arxiv.org/html/2608.16927#bib.bib24); Biet al\.[2025](https://arxiv.org/html/2608.16927#bib.bib17)\)\. Some methods further incorporate quality or importance signals to improve the effectiveness of the selected subset\(Leeet al\.[2024](https://arxiv.org/html/2608.16927#bib.bib19); Yanet al\.[2025](https://arxiv.org/html/2608.16927#bib.bib23)\)\. Although these methods are effective in practice, most of them still rely on geometric relations in the original embedding space, implicitly assuming that neighborhood relations and aggregation patterns in this space are aligned with the semantic coverage and supervision differences required for data selection\.

However, we find that this assumption is insufficient\. For complex instruction data, geometric relations in the original embedding space do not only reflect dominant semantic directions, but are also affected by fine\-grained attributes within these directions, response format variations, and local noise\(Chenet al\.[2025](https://arxiv.org/html/2608.16927#bib.bib27)\)\. Our embedding geometry analysis further shows that the data distribution has a high degree of global variation and is difficult to summarize with a small number of linear directions\. In contrast, local neighborhoods exhibit substantially lower effective dimensionality, and their neighborhood structure is more consistent with a nonlinear manifold assumption, as shown in Appendix[A](https://arxiv.org/html/2608.16927#A1)\. Therefore, performing flat diversity measurement directly in the original embedding space may mix global semantic structure, local fine\-grained differences, and task irrelevant perturbations into a single selection criterion\. Based on this observation, we argue that data selection should be formulated as a coarse\-to\-fine hierarchical coverage problem: the selected subset should first preserve coverage over major semantic manifold regions, and then further cover fine grained supervision features within each region\. To this end, we propose MASS, a manifold aware sparse selection method\. MASS first uses a dense autoencoder to learn stable low\-dimensional principal manifold coordinates, and performs coarse grained grouping in this space\. It then uses a TopK sparse autoencoder to extract sparse features, and performs fine grained coverage selection within each coarse grained group\. In addition, MASS incorporates external quality scores to avoid selecting low quality samples merely for increasing diversity\.

We evaluate MASS on Vision\-Flan and LLaVA\-CoT, covering both general instruction and reasoning tasks\. Experimental results show that MASS consistently outperforms existing data selection baselines under multiple data budgets, and in some settings even surpasses full data training performance using only a small subset of data\. Further ablation studies show that the principal manifold coordinates learned by the DAE provide more reliable coarse grained grouping, the sparse features learned by the SAE improve fine\-grained coverage within each group, and quality signals further enhance the effectiveness of the selected samples\.

Our main contributions are summarized as follows:

- •We reformulate supervised fine\-tuning data selection as a hierarchical coverage problem that combines coarse\-grained principal manifold coverage with fine\-grained supervision feature coverage\.
- •We propose MASS, which constructs principal manifold groups with a DAE, performs within\-group sparse feature coverage with a TopK SAE, and incorporates quality signals for sample selection\.
- •We validate MASS across multiple datasets, data budgets, embedding sources, and target models, demonstrating its effectiveness and robustness\.

## 2Related Work

#### Data Selection\.

Data selection aims to identify a high\-value subset from large\-scale training data, reducing training cost while preserving or even improving model performance\. Existing methods usually select data from the perspective of sample importance or coverage diversity: the former includes methods based on training dynamics, such as EL2N\(Paulet al\.[2021](https://arxiv.org/html/2608.16927#bib.bib14)\), LESS\(Xiaet al\.[2024](https://arxiv.org/html/2608.16927#bib.bib12)\), and OPUS\(Wanget al\.[2026](https://arxiv.org/html/2608.16927#bib.bib13)\), as well as methods based on a model’s own behavior or performance, such as ScalSelect\(Wuet al\.[2026](https://arxiv.org/html/2608.16927#bib.bib15)\)and CVS\(Sunet al\.[2026](https://arxiv.org/html/2608.16927#bib.bib21)\); the latter includes representation similarity based methods, such as SemDeDup\(Abbaset al\.[2023](https://arxiv.org/html/2608.16927#bib.bib16)\)and PRISM\(Biet al\.[2025](https://arxiv.org/html/2608.16927#bib.bib17)\), and diversity consensus based methods, such as ICONS\(Wuet al\.[2024](https://arxiv.org/html/2608.16927#bib.bib22)\)and INSTAG\(Luet al\.[2023](https://arxiv.org/html/2608.16927#bib.bib18)\)\. In addition, methods such as COINCIDE\(Leeet al\.[2024](https://arxiv.org/html/2608.16927#bib.bib19)\)and XMAS\(Naharaset al\.[2025](https://arxiv.org/html/2608.16927#bib.bib20)\)further combine sample importance with data diversity, and have shown more stable selection performance in practice\. Although effective, these methods mostly rely on global scores or distribution level similarity measures, whereas MASS formulates data selection as a coarse\-to\-fine hierarchical coverage problem that preserves principal manifold coverage, models local fine\-grained supervision feature coverage, and uses sample quality as an auxiliary signal\.

#### Autoencoders and Manifold Representation\.

Autoencoders compress high\-dimensional inputs into a low\-dimensional latent space through an architecture with an encoder and a decoder, and learn compact representations via a reconstruction objective\(Baldi and Hornik[1989](https://arxiv.org/html/2608.16927#bib.bib1); Hinton and Salakhutdinov[2006](https://arxiv.org/html/2608.16927#bib.bib3)\)\. Previous studies show that low\-dimensional latent representations can preserve the principal directions of variation in data, and thus have been widely used for nonlinear dimensionality reduction, representation learning, and manifold structure modeling\(Kramer[1991](https://arxiv.org/html/2608.16927#bib.bib2); Hinton and Salakhutdinov[2006](https://arxiv.org/html/2608.16927#bib.bib3); Bengioet al\.[2013](https://arxiv.org/html/2608.16927#bib.bib4)\)\. In MASS, we adopt a dense autoencoder \(DAE\) to compress condition representations and learn low\-dimensional principal manifold coordinates, which are used to construct stable coarse\-grained semantic groups\. Compared with clustering directly in the original embedding space, the bottleneck structure of DAE helps suppress redundant dimensions and local perturbations, thereby better characterizing the dominant structure of the data distribution\.

#### Sparse Autoencoders\.

Sparse autoencoders originate from sparse coding and dictionary learning, aiming to reconstruct input representations with a small number of activated features and thereby obtain more selective and interpretable latent features\(Olshausen and Field[1997](https://arxiv.org/html/2608.16927#bib.bib5); Aharonet al\.[2006](https://arxiv.org/html/2608.16927#bib.bib6)\)\. Recently, SAEs have been widely used for mechanistic interpretability of language models, where they decompose internal model activations into more semantically coherent features\(Hubenet al\.[2024](https://arxiv.org/html/2608.16927#bib.bib7); Brickenet al\.[2023](https://arxiv.org/html/2608.16927#bib.bib8)\)\. Subsequent works further propose TopK SAE\(Makhzani and Frey[2013](https://arxiv.org/html/2608.16927#bib.bib9)\), Gated SAE\(Rajamanoharanet al\.[2024a](https://arxiv.org/html/2608.16927#bib.bib10)\), and JumpReLU SAE\(Rajamanoharanet al\.[2024b](https://arxiv.org/html/2608.16927#bib.bib11)\)to improve the tradeoff among sparsity control, dead features, and reconstruction quality\. Unlike prior works that mainly focus on interpreting internal model activations, MASS applies TopK SAE to joint input response supervision representations and uses sparse feature coverage gain for data selection, thereby improving the diversity of fine\-grained supervision signals within each principal manifold region\.

## 3Method

### 3\.1Problem Formulation

Given a datasetDD, data selection seeks a budgeted subsetD′⊆DD^\{\\prime\}\\subseteq Dwith\|D′\|=P\|D^\{\\prime\}\|=P, such that training onD′D^\{\\prime\}approaches or even surpasses the performance of training onDD\. Letf​\(⋅\)f\(\\cdot\)denote the resulting model performance\. We formulate the objective as

maxD′⊆Df\(D′\)s\.t\.\|D′\|=P\.\\max\_\{D^\{\\prime\}\\subseteq D\}f\(D^\{\\prime\}\)\\quad\\mathrm\{s\.t\.\}\\quad\|D^\{\\prime\}\|=P\.\(1\)

### 3\.2MASS: Manifold Aware Sparse Selection

We propose MASS, a manifold aware sparse selection method for SFT data selection\. MASS uses a dense autoencoder \(DAE\) to learn low\-dimensional principal manifold coordinates for coarse semantic grouping, and a TopK sparse autoencoder \(SAE\) to capture fine grained sparse supervision features\. It then performs greedy selection within each group by combining sparse feature coverage with external quality scores, preserving manifold coverage while improving fine grained diversity and sample quality\. The overall pipeline is shown in Figure[1](https://arxiv.org/html/2608.16927#S3.F1), with implementation details provided in Appendix[B](https://arxiv.org/html/2608.16927#A2)\.

#### Dual View Encoding\.

Given a candidate sampleziz\_\{i\}, we represent it as an input and response pair:

zi=\(qi,ai\),z\_\{i\}=\(q\_\{i\},a\_\{i\}\),\(2\)whereqiq\_\{i\}denotes the input, which may contain textual instructions, visual information, or other contextual content, andaia\_\{i\}denotes the response\. For each sample, we construct two types of embeddings\. The first one is the condition embedding:

eic=Eemb​\(qi\),e\_\{i\}^\{c\}=E\_\{\\mathrm\{emb\}\}\(q\_\{i\}\),\(3\)which encodes only the semantic information contained in the input, and is used for subsequent DAE manifold coordinate learning and coarse grained grouping\. The second one is the supervision embedding:

eis=Eemb​\(qi,ai\),e\_\{i\}^\{s\}=E\_\{\\mathrm\{emb\}\}\(q\_\{i\},a\_\{i\}\),\(4\)which encodes the complete supervision signal jointly formed by the input and the response, and is used for subsequent SAE sparse feature learning and fine grained coverage within each group\. Both embeddings are extracted with a frozen embedding model, then independently centered andL2L\_\{2\}\-normalized\. In the following sections,eice\_\{i\}^\{c\}andeise\_\{i\}^\{s\}denote the preprocessed embeddings\.

![Refer to caption](https://arxiv.org/html/2608.16927v1/images/mass.jpg)Figure 1:Overview of MASS\. MASS first uses a DAE to learn low\-dimensional principal manifold coordinates from input side representations and performs coarse grained grouping\. It then uses a TopK SAE to extract sparse supervision features from input response representations\. Finally, within each group, MASS greedily selects samples by combining feature coverage gain with quality scores, producing the final selected subset\.
#### DAE for Principal Manifold Encoding\.

We train a DAE in the condition embedding space to map high\-dimensional embeddings into stable low\-dimensional principal manifold coordinates\. Specifically, during training, giveneice\_\{i\}^\{c\}, we construct a perturbation with a fixed magnitude:

ui=ϵi‖ϵi‖2,ϵi∼𝒩​\(0,I\)\.u\_\{i\}=\\frac\{\\epsilon\_\{i\}\}\{\\\|\\epsilon\_\{i\}\\\|\_\{2\}\},\\quad\\epsilon\_\{i\}\\sim\\mathcal\{N\}\(0,I\)\.\(5\)We then apply this perturbation toeice\_\{i\}^\{c\}and normalize the result as the DAE input:

e~ic=Norm​\(eic\+δ​ui\),\\widetilde\{e\}\_\{i\}^\{c\}=\\mathrm\{Norm\}\\bigl\(e\_\{i\}^\{c\}\+\\delta u\_\{i\}\\bigr\),\(6\)whereδ\\deltais a small fixed constant, andNorm​\(⋅\)\\mathrm\{Norm\}\(\\cdot\)denotesL2L\_\{2\}normalization\. The DAE consists of an encoderϕdae\\phi\_\{\\mathrm\{dae\}\}and a decoderψdae\\psi\_\{\\mathrm\{dae\}\}\.ϕdae\\phi\_\{\\mathrm\{dae\}\}compresses the perturbed high\-dimensional representation into a low\-dimensional latent representation with dimension 32:

hi=ϕdae​\(e~ic\),h\_\{i\}=\\phi\_\{\\mathrm\{dae\}\}\(\\widetilde\{e\}\_\{i\}^\{c\}\),\(7\)wherehih\_\{i\}is regarded as the low\-dimensional principal manifold coordinate of the sample in the condition embedding space\.ψdae\\psi\_\{\\mathrm\{dae\}\}then reconstructs this coordinate back into the original embedding space:

e^ic=Norm​\(ψdae​\(hi\)\)\.\\hat\{e\}\_\{i\}^\{c\}=\\mathrm\{Norm\}\\bigl\(\\psi\_\{\\mathrm\{dae\}\}\(h\_\{i\}\)\\bigr\)\.\(8\)The training objective is to minimize the squared distance between the original condition embedding and the DAE reconstruction:

ℒDAE=1N​∑i=1N‖eic−e^ic‖22\.\\mathcal\{L\}\_\{\\mathrm\{DAE\}\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\left\\\|e\_\{i\}^\{c\}\-\\hat\{e\}\_\{i\}^\{c\}\\right\\\|\_\{2\}^\{2\}\.\(9\)

#### SAE for Sparse Feature Encoding\.

We train a TopK SAE in the supervision embedding space to characterize fine\-grained sparse features in complete samples\. Unlike the DAE, the SAE takeseise\_\{i\}^\{s\}as input, since the effective training signal of a sample comes not only from the inputqiq\_\{i\}, but also from the responseaia\_\{i\}\. Specifically, the SAE consists of an encoderϕsae\\phi\_\{\\mathrm\{sae\}\}and a decoderψsae\\psi\_\{\\mathrm\{sae\}\}\. Given the supervision representationeise\_\{i\}^\{s\},ϕsae\\phi\_\{\\mathrm\{sae\}\}first produces an overcomplete feature preactivation, which is then passed through a ReLU nonlinearity to obtain nonnegative activations:

ri=ReLU​\(ϕsae​\(eis\)\)\.r\_\{i\}=\\mathrm\{ReLU\}\\bigl\(\\phi\_\{\\mathrm\{sae\}\}\(e\_\{i\}^\{s\}\)\\bigr\)\.\(10\)We then apply the TopK operation for explicit sparsification:

ki=TopK​\(ri,K\),k\_\{i\}=\\mathrm\{TopK\}\(r\_\{i\},K\),\(11\)wherekik\_\{i\}is a sparse vector with at mostKKnonzero activations\. In the main experiments, we set the dimension of nonnegative feature activations to 131072 and useK=64K=64for the TopK operation\.ψsae\\psi\_\{\\mathrm\{sae\}\}then reconstructs the supervision embedding from the sparse vector:

e^is=Norm​\(ψsae​\(ki\)\)\.\\hat\{e\}\_\{i\}^\{s\}=\\mathrm\{Norm\}\(\\psi\_\{\\mathrm\{sae\}\}\(k\_\{i\}\)\)\.\(12\)The training objective is to minimize the squared distance between the original supervision embedding and the SAE reconstruction:

ℒSAE=1N​∑i=1N‖eis−e^is‖22\.\\mathcal\{L\}\_\{\\mathrm\{SAE\}\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\left\\\|e\_\{i\}^\{s\}\-\\hat\{e\}\_\{i\}^\{s\}\\right\\\|\_\{2\}^\{2\}\.\(13\)
![Refer to caption](https://arxiv.org/html/2608.16927v1/images/main_experiment2.png)Figure 2:Main results on Vision\-Flan and LLaVA\-CoT under different data budgets\. We report the average relative performance \(ARP\) of MASS and representative baselines under 5%, 10%, and 15% budgets\. MASS achieves consistently strong performance across budgets, outperforming existing baselines on both datasets and surpassing full\-data training in several settings\.
#### Coarse to Fine Selection\.

After training, we first extract DAE latents from clean condition embeddings and perform KMeans clustering, obtaining coarse\-grained groups\{Gl\}\\\{\{G\}\_\{l\}\\\}\. For a selection ratioρ\\rho, each groupGl\{G\}\_\{l\}is assigned a sampling budget:

Bl=⌊ρ​\|Gl\|⌋\.B\_\{l\}=\\lfloor\\rho\|\{G\}\_\{l\}\|\\rfloor\.\(14\)We then sequentially select samples within eachGl\{G\}\_\{l\}\. For sampleii, we define its activated feature set as

Ωi=\{j:ki​j\>0\}\.\\Omega\_\{i\}=\\\{j:k\_\{ij\}\>0\\\}\.\(15\)At selection steptt, letSl,t⊆GlS\_\{l,t\}\\subseteq\{G\}\_\{l\}denote the current selected subset\. The SAE coverage gain of candidate sampleiiis defined as

ΔSAE​\(i∣Sl,t\)=1\|Ωi\|​∑j∈Ωiki​jMj\+ϵ⋅1nl,t​\(j\)\+1,\\Delta\_\{\\mathrm\{SAE\}\}\(i\\mid S\_\{l,t\}\)=\\frac\{1\}\{\|\\Omega\_\{i\}\|\}\\sum\_\{j\\in\\Omega\_\{i\}\}\\frac\{k\_\{ij\}\}\{M\_\{j\}\+\\epsilon\}\\cdot\\frac\{1\}\{n\_\{l,t\}\(j\)\+1\},\(16\)where

Mj\\displaystyle M\_\{j\}=maxi⁡ki​j,\\displaystyle=\\max\_\{i\}k\_\{ij\},\(17\)nl,t​\(j\)\\displaystyle n\_\{l,t\}\(j\)=∑m∈Sl,t𝟏​\{j∈Ωm\}\.\\displaystyle=\\sum\_\{m\\in S\_\{l,t\}\}\\mathbf\{1\}\\\{j\\in\\Omega\_\{m\}\\\}\.Here,MjM\_\{j\}denotes the maximum observed activation of SAE featurejjand is used to normalize feature scales, whilenl,t​\(j\)n\_\{l,t\}\(j\)counts how many times featurejjhas been covered in the current subset\. We combine the SAE coverage gain with an external quality scoreηi∈\[0,1\]\\eta\_\{i\}\\in\[0,1\]and greedily select the highest scoring remaining candidate in groupGl\{G\}\_\{l\}:

it\(l\)=arg⁡maxi∈Gl∖Sl,t⁡\(ΔSAE​\(i∣Sl,t\)\+ηi\)\.i\_\{t\}^\{\(l\)\}=\\arg\\max\_\{i\\in\{G\}\_\{l\}\\setminus S\_\{l,t\}\}\\left\(\\Delta\_\{\\mathrm\{SAE\}\}\(i\\mid S\_\{l,t\}\)\+\\eta\_\{i\}\\right\)\.\(18\)The selected sample is then added to the current subset:

Sl,t\+1=Sl,t∪\{it\(l\)\}\.S\_\{l,t\+1\}=S\_\{l,t\}\\cup\\\{i\_\{t\}^\{\(l\)\}\\\}\.\(19\)This greedy procedure is repeated until\|Sl,t\|=Bl\|S\_\{l,t\}\|=B\_\{l\}, yielding the selected subsetSlS\_\{l\}for groupll\. The final output set is

S=⋃lSl\.S=\\bigcup\_\{l\}S\_\{l\}\.\(20\)

## 4Experiments

### 4\.1Experimental Setup

#### Datasets\.

We evaluate MASS on Vision\-Flan\(Xuet al\.[2024](https://arxiv.org/html/2608.16927#bib.bib29)\)for general instruction tasks and LLaVA\-CoT\(Xuet al\.[2025](https://arxiv.org/html/2608.16927#bib.bib30)\)for reasoning tasks, covering different task complexities and data distributions\. Dataset details and preprocessing are provided in Appendix[C](https://arxiv.org/html/2608.16927#A3)\.

#### Target Models\.

We use LLaVA\-V1\.5\-7B\(Liuet al\.[2024a](https://arxiv.org/html/2608.16927#bib.bib31)\)as the target model on Vision\-Flan and Llama\-3\.2\-11B\-Vision\-Instruct\(Grattafioriet al\.[2024](https://arxiv.org/html/2608.16927#bib.bib32)\)on LLaVA\-CoT\. Detailed training hyperparameters are provided in Appendix[D](https://arxiv.org/html/2608.16927#A4)\.

#### Data Budgets\.

We evaluate MASS under three data budgets, 5%, 10%, and 15%, for both datasets\.

#### Baselines\.

We compare MASS against 9 data selection baselines, including Random Selection, XMAS\(Naharaset al\.[2025](https://arxiv.org/html/2608.16927#bib.bib20)\), COINCIDE\(Leeet al\.[2024](https://arxiv.org/html/2608.16927#bib.bib19)\), SemDeDup\(Abbaset al\.[2023](https://arxiv.org/html/2608.16927#bib.bib16)\), D2 Pruning\(Maharanaet al\.[2023](https://arxiv.org/html/2608.16927#bib.bib33)\), PRISM\(Biet al\.[2025](https://arxiv.org/html/2608.16927#bib.bib17)\), ScalSelect\(Wuet al\.[2026](https://arxiv.org/html/2608.16927#bib.bib15)\), CLIP Score\(Hesselet al\.[2021](https://arxiv.org/html/2608.16927#bib.bib34)\), and EL2N\(Paulet al\.[2021](https://arxiv.org/html/2608.16927#bib.bib14)\)\. These baselines cover representative data selection paradigms based on importance estimation, diversity distribution, and the combination of both perspectives\.

#### Quality and Embedding Models\.

To reduce computational overhead, we perform scoring with vLLM\(Kwonet al\.[2023](https://arxiv.org/html/2608.16927#bib.bib39)\)using lightweight models\. Specifically, we use Qwen3\-VL\-4B\-Instruct\(Baiet al\.[2025](https://arxiv.org/html/2608.16927#bib.bib35)\)for quality scoring on Vision\-Flan and Qwen3\.5\-9B\(Team[2026](https://arxiv.org/html/2608.16927#bib.bib36)\)on LLaVA\-CoT\. For embedding extraction, we use Qwen3\-VL\-Embedding\-2B\(Liet al\.[2026](https://arxiv.org/html/2608.16927#bib.bib37)\)for both datasets\.

#### Evaluation\.

Following the intended use of each dataset, we evaluate the trained target models on different benchmark suites\. Vision\-Flan and LLaVA\-CoT are each evaluated on 12 benchmarks: Vision\-Flan focuses on general capability evaluation, while LLaVA\-CoT covers both reasoning and general capabilities\. The detailed benchmark lists and full results are provided in Appendix[E](https://arxiv.org/html/2608.16927#A5)\. To ensure stable results, we run all experiments with three random seeds, 0, 42, and 99, and report the average performance\. To normalize scores across benchmarks and datasets, we report Average Relative Performance \(ARP\):

ARP=Subset Data PerformanceFull Data Performance×100\.\\mathrm\{ARP\}=\\frac\{\\text\{Subset Data Performance\}\}\{\\text\{Full Data Performance\}\}\\times 100\.\(21\)

### 4\.2Main Results

Figure[2](https://arxiv.org/html/2608.16927#S3.F2)presents the results of MASS on the Vision\-Flan and LLaVA\-CoT datasets\.

On Vision\-Flan, MASS achieves ARP of 99\.50, 102\.12, and 103\.23 under the 5%, 10%, and 15% budgets, respectively, outperforming all baselines\. In particular, under the 10% and 15% budgets, MASS not only surpasses the strongest baseline but also exceeds Full Data by 2\.12 and 3\.23 points, respectively\. On LLaVA\-CoT, MASS also achieves the best performance across all budget settings, with ARP of 99\.65, 99\.83, and 102\.82\. It is worth noting that several baselines on this dataset already approach or even surpass Full Data\. For example, EL2N reaches 102\.46 under the 15% budget, while MASS further improves upon it and obtains the highest performance\.

Overall, MASS consistently achieves the best results on both datasets and across all three data budgets, demonstrating its stable and effective data selection capability under different data distributions and task complexities\. These results validate our central hypothesis: effective data selection should not be restricted to global sampling in the original embedding space\. Instead, it should be formulated as a hierarchical coverage problem that preserves the principal manifold distribution of the data while also covering locally fine grained features\.

## 5Analysis and Ablation Studies

### 5\.1Effect of DAE Manifold Coordinates

To verify whether coarse grouping requires DAE manifold coordinates, we compare MASS with a variant on Vision\-Flan under the 10% budget, where DAE clustering is replaced by clustering in the original embedding space\. All other settings are kept unchanged\.

As shown in Table[1](https://arxiv.org/html/2608.16927#S5.T1), clustering based on raw embeddings achieves an ARP of 101\.59, whereas clustering based on the DAE principal manifold coordinates achieves a higher ARP of 102\.12\. This result suggests that although raw embeddings contain rich semantic information, their geometric structure may still mix redundant dimensions, local perturbations, and non dominant variations, leading to less stable coarse grained grouping\. In contrast, the low\-dimensional principal manifold coordinates learned by DAE can more effectively capture the principal semantic directions of the data, providing more reliable coarse grained region partitions for subsequent SAE based coverage selection within each cluster and thereby improving the final performance\.

### 5\.2Complementarity of DAE and SAE

To analyze the complementarity of DAE and SAE, we construct two variants on Vision\-Flan under the 10% budget\. The first removes SAE and uses only DAE clustering followed by random sampling within each cluster\. The second removes DAE and directly performs fine granularity coverage selection based on SAE features\.

As shown in Figure[3](https://arxiv.org/html/2608.16927#S5.F3), random selection obtains an ARP of 95\.30, while the variant using only DAE reaches 97\.71, indicating that principal manifold grouping provides useful coverage at a coarse granularity\. The variant using only SAE achieves an ARP of 101\.44, showing that fine granularity feature coverage can effectively improve the quality of sample coverage\. The full MASS achieves the best ARP of 102\.12, outperforming both single module variants\. These results demonstrate that DAE and SAE are complementary at two levels of coverage: DAE maintains coarse granularity coverage over the principal manifold, while SAE enhances fine granularity feature coverage within each manifold region\.

Table 1:Effect of DAE manifold coordinates\. DAE clustering improves ARP over raw embedding clustering, showing the benefit of principal manifold coordinates for coarse grouping\.![Refer to caption](https://arxiv.org/html/2608.16927v1/images/fig_ablation.png)Figure 3:Complementarity of DAE and SAE\. DAE alone improves over random selection by preserving coarse manifold coverage, while SAE alone brings a larger gain through fine grained feature coverage\. Combining both components yields the best performance, demonstrating the complementarity between DAE and SAE
### 5\.3Sensitivity to DAE Configuration

To analyze the effect of DAE configuration on MASS, we vary the DAE latent dimension on Vision\-Flan under the 10% budget, while keeping all other settings unchanged\.

As shown in Figure[4](https://arxiv.org/html/2608.16927#S5.F4), when the latent dimension is set to 8, 16, 32, 64, and 128, MASS achieves ARP of 100\.66, 98\.86, 102\.12, 102\.71, and 101\.77, respectively\. The results show that either too small or too large a DAE latent dimension can hurt the final performance\. When the dimension is too small, the principal manifold coordinates may fail to preserve sufficient semantic structure, leading to inadequate coarse granularity grouping\. When the dimension is too large, the embedding space may reintroduce redundant dimensions and local perturbations, weakening the ability of DAE to extract the principal semantic directions\. Overall, the 64 dimensional setting achieves the best performance, suggesting that a moderate compression dimension better balances principal manifold structure preservation and suppression of non dominant variations\.

### 5\.4Sensitivity to SAE Configuration

To analyze the effect of SAE configuration on MASS, we study both the SAE feature dimension and the TopK sparsity on Vision\-Flan\. For the feature dimension analysis, we compare SAE dimensions of 65536 and 131072 under 5%, 10%, and 15% budgets\. For the TopK analysis, we fix the SAE feature dimension to 131072 and vary the TopK value under the 10% data budget\. All other settings are kept unchanged\.

![Refer to caption](https://arxiv.org/html/2608.16927v1/images/fig_dimension_arp1.png)Figure 4:Sensitivity to DAE configuration\. Performance drops when the latent dimension is either too small or too large, while a moderate dimension of 64 achieves the best ARP, suggesting the importance of proper latent capacity\.![Refer to caption](https://arxiv.org/html/2608.16927v1/images/topk_arp1.png)Figure 5:Effect of TopK sparsity\. MASS achieves the best ARP with TopK of 64, while both smaller and larger TopK values lead to lower performance, indicating the importance of a proper sparsity level\.#### Effect of TopK Sparsity\.

As shown in Figure[5](https://arxiv.org/html/2608.16927#S5.F5), when TopK is set to 16, 32, 64, and 128, MASS achieves ARP of 100\.81, 100\.11, 102\.12, and 101\.21, respectively\. These results suggest that a too small TopK limits the number of activated features for each sample, resulting in insufficient fine granularity coverage\. In contrast, a too large TopK weakens the sparsity of SAE representations and may introduce more non essential features\. TopK of 64 achieves the best result, indicating that a moderate sparsity level better balances feature coverage capacity and representation selectivity\.

#### Effect of SAE Feature Dimension\.

As shown in Figure[6](https://arxiv.org/html/2608.16927#S5.F6), when the SAE feature dimension is 65536, MASS achieves ARP of 99\.19, 100\.98, and 102\.81 under the 5%, 10%, and 15% budgets, respectively\. Increasing the feature dimension to 131072 improves the corresponding ARP to 99\.50, 102\.12, and 103\.23\. These results indicate that a larger SAE feature space provides stronger capacity for fine granularity feature representation, thereby improving the effectiveness of within cluster sparse feature coverage\.

![Refer to caption](https://arxiv.org/html/2608.16927v1/images/sae_dimension.png)Figure 6:Effect of SAE feature dimension\. Increasing the SAE feature dimension from 65536 to 131072 consistently improves ARP across data budgets, suggesting the importance of sufficient sparse feature capacity\.

### 5\.5Effect of Quality Signals

To analyze the effect of external quality scores on MASS, we further test two settings on Vision\-Flan under the 10% data budget: removing the quality score and replacing the scoring model with LLaVA\-OneVision\-1\.5\-4B\-Instruct\(Anet al\.[2025](https://arxiv.org/html/2608.16927#bib.bib40)\)\. All other settings are kept unchanged\.

As shown in Table[2](https://arxiv.org/html/2608.16927#S5.T2), random selection obtains an ARP of only 95\.30, while MASS without quality scores reaches 99\.96\. This indicates that the hierarchical coverage mechanism based on DAE and SAE already provides a strong data selection effect\. After adding external quality scores, MASS achieves 102\.12 ARP with the original scoring model and 102\.44 ARP with LLaVA\-OneVision\-1\.5\-4B\-Instruct\. These results show that external quality scores can effectively filter low quality samples and complement both coarse granularity principal manifold coverage and fine granularity feature coverage\. Moreover, different scoring models bring consistent improvements, suggesting that MASS is not overly sensitive to the specific choice of quality model\.

Table 2:Effect of quality signals\. MASS without quality signals already improves over random selection, while different scoring models further increase ARP, showing that quality signals are useful and robust across scorers\.Table 3:Effect of embedding source\. MASS remains effective with GME embeddings, substantially outperforming random selection\.
### 5\.6Effect of Embedding Source

To analyze the effect of the embedding source on MASS, we replace the original embedding source with GME\(Zhanget al\.[2024](https://arxiv.org/html/2608.16927#bib.bib41)\)on Vision\-Flan under the 10% data budget, while keeping all other settings unchanged\.

As shown in Table[3](https://arxiv.org/html/2608.16927#S5.T3), random selection obtains an ARP of 95\.30, while MASS with GME embeddings achieves 101\.68, substantially outperforming random selection\. This result shows that the hierarchical selection mechanism of MASS does not rely on a single embedding source and can still maintain strong performance after changing the embedding representation\. Meanwhile, the result with GME is slightly lower than the 102\.12 ARP obtained with the original embedding source, which is consistent with the relatively weaker representation ability of GME in our setting\. This suggests that MASS has certain adaptability to different embedding sources, while a stronger embedding representation can still provide a better basis for DAE principal manifold grouping and SAE fine granularity feature coverage\.

![Refer to caption](https://arxiv.org/html/2608.16927v1/images/model_fitting.png)Figure 7:Robustness across target models\. We evaluate MASS by replacing the target models with LLaVA\-v1\.5\-13B on Vision\-Flan and Qwen2VL\-2B\-Instruct on LLaVA\-CoT, the results show that MASS maintains stable performance across different model scales and model families
### 5\.7Robustness Across Target Models

To evaluate the robustness of MASS across target models, we test LLaVA\-v1\.5\-13B\(Liuet al\.[2024a](https://arxiv.org/html/2608.16927#bib.bib31)\)on Vision\-Flan and Qwen2VL\-2B\-Instruct\(Wanget al\.[2024b](https://arxiv.org/html/2608.16927#bib.bib38)\)on LLaVA\-CoT, covering both variation in parameter scale within the same model family and variation across different model families\. All other settings are kept unchanged\.

As shown in Figure[7](https://arxiv.org/html/2608.16927#S5.F7), on Vision\-Flan, MASS achieves ARP of 89\.49, 93\.09, and 94\.15 under the 5%, 10%, and 15% budgets, respectively, outperforming random selection with 85\.12, 92\.26, and 91\.14\. On LLaVA\-CoT, MASS achieves 96\.52, 96\.02, and 95\.36 ARP under the three budgets, also consistently outperforming random selection with 94\.30, 95\.25, and 94\.77\. These results show that MASS remains effective under different target model scales and model families, demonstrating its robustness across target models\.

## 6Conclusion and Limitations

#### Conclusion\.

This paper formulates supervised fine tuning data selection as a coarse to fine hierarchical coverage problem\. To address the entanglement of principal semantic directions, fine grained supervision differences, and local noise in the original embedding space, we propose MASS\. MASS uses a dense autoencoder to learn low\-dimensional principal manifold coordinates for coarse grained grouping, and then performs fine grained selection within each group by combining TopK sparse autoencoder features with quality signals\. Experimental results show that MASS consistently outperforms existing baselines across multiple datasets, data budgets, embedding sources, and target models, validating the effectiveness of principal manifold coverage and within group sparse feature coverage\. MASS provides a hierarchical coverage paradigm for embedding based data selection\.

#### Limitations\.

Despite its stable and effective performance, MASS has several limitations\. First, MASS relies on external embedding source and quality scoring models, so the selected subset may be affected when these external signals are severely biased\. Second, MASS focuses on hierarchical coverage selection within a given candidate pool; when the pool itself lacks certain task types or supervision patterns, the method cannot compensate for such fundamental distributional gaps\. Finally, MASS does not study how to select subsets from general data collections for specific downstream tasks, leaving its applicability to task oriented data selection for future exploration\.

## Appendix AEmbedding Geometry Analysis

To better understand whether the original embedding space is suitable for direct data selection, we analyze the embedding geometry of Vision\-Flan and LLaVA\-CoT\. Both datasets use the same embedding preprocessing procedure as MASS, where embeddings are first centered and thenL2L\_\{2\}normalized\. Table[4](https://arxiv.org/html/2608.16927#A6.T4)summarizes several geometry statistics\. The number of PCA components required to explain 90% of the variance and the PCA effective rank are used to characterize the global linear complexity of the embedding distribution\. The TwoNN intrinsic dimension is used to estimate the intrinsic dimensionality of the data\. The local PCA dimension estimates, for each sample neighborhood, the minimum number of local principal components required to explain 90% of the local variance\. Finally, we compare the neighborhood preservation of PCA and Isomap to measure how well low\-dimensional representations preserve the local neighborhood structure of the original embedding space\.

The results show that the embedding distributions of both datasets have high global linear complexity\. Although the embedding dimension of both Vision\-Flan and LLaVA\-CoT is 2048, Vision\-Flan still requires 379 PCA components to explain 90% of the variance, while LLaVA\-CoT requires 459 components\. Meanwhile, their PCA effective ranks reach 255\.06 and 275\.17, respectively\. These results indicate that the data are not concentrated in a simple low\-dimensional linear subspace, and that global distances and aggregation patterns in the original embedding space may be affected by many variation directions\.

At the same time, the intrinsic dimension and local PCA results show that the data exhibit much lower effective dimensionality in local neighborhoods\. The TwoNN intrinsic dimension of Vision\-Flan is 8\.25, and its mean and median local PCA dimensions are 17\.38 and 19, respectively\. The TwoNN intrinsic dimension of LLaVA\-CoT is 11\.88, and its mean and median local PCA dimensions are 20\.88 and 21, respectively\. The large gap between global PCA complexity and local dimensionality suggests that instruction data embeddings are not unstructured high\-dimensional point clouds\. Instead, they are better characterized as globally complex distributions with locally low\-dimensional geometry\.

Furthermore, the comparison between PCA and Isomap neighborhood preservation indicates that this local structure is nonlinear\. On Vision\-Flan, Isomap achieves a neighborhood preservation score of 0\.499, which is higher than the PCA score of 0\.383\. On LLaVA\-CoT, Isomap also improves the score from 0\.281 under PCA to 0\.380\. Since PCA captures a global linear projection while Isomap better preserves nonlinear neighborhood geometry, this result suggests that local neighborhood relations in the embedding space are more consistent with a nonlinear manifold structure than with a single global linear structure\.

These observations support the central motivation of this work\. The original embedding space of complex instruction data contains global semantic variation, local fine grained differences, and task irrelevant perturbations at the same time\. If data selection is performed directly in this space, a single distance or diversity criterion may fail to distinguish these different levels of variation\. Based on this observation, MASS formulates data selection as a coarse to fine hierarchical coverage process\. It first uses a DAE to learn stable low\-dimensional principal manifold coordinates for constructing coarse grained semantic regions, and then uses an SAE to extract sparse supervision features for fine grained coverage selection within each region\.

## Appendix BImplementation Details of MASS

This section supplements the implementation details and hyperparameters omitted from Section[3\.2](https://arxiv.org/html/2608.16927#S3.SS2)to ensure experimental reproducibility\.

#### Dual View Encoding\.

For each samplezi=\(qi,ai\)z\_\{i\}=\(q\_\{i\},a\_\{i\}\), the condition embeddingeice\_\{i\}^\{c\}is obtained by feeding onlyqiq\_\{i\}into the frozen embedding model, while the supervision embeddingeise\_\{i\}^\{s\}is obtained by feeding the concatenated pair\(qi,ai\)\(q\_\{i\},a\_\{i\}\)into the embedding model\.

#### Dense Autoencoder for Principal Manifold Encoding\.

The dense autoencoder is used to learn low\-dimensional principal manifold coordinates from input side condition embeddings\. During training, input embeddings are firstL2L\_\{2\}normalized\. We then add spherical noise with a fixed magnitude of 0\.03 and applyL2L\_\{2\}normalization again before feeding them into the model\. The model is a simple MLP autoencoder\. Its encoder has hidden dimensions of 1024, 512, and 128 with GELU activations, and maps each input to a low\-dimensional latent space\. The decoder uses a symmetric architecture to reconstruct the latent representation back to the original embedding space\. In main experiments, latent dimension is set to 32\. We train the dense autoencoder with AdamW using a learning rate of1×10−41\\times 10^\{\-4\}, a weight decay of1×10−41\\times 10^\{\-4\}, a batch size of 128, and 200 epochs\. We use 2% of data for validation and select the best checkpoint based on validation reconstruction loss\. After training, clean condition embeddings are encoded into latent coordinates, and KMeans clustering with 256 clusters is applied for coarse grained grouping\.

#### Sparse Autoencoder for Sparse Feature Encoding\.

The sparse autoencoder is used to model fine grained sparse features in the supervision embeddings formed jointly by the input and the response\. Before training, we compute the training set mean, center supervision embeddings, and applyL2L\_\{2\}normalization\. We use a TopK sparse autoencoder\. Its encoder maps each input into an overcomplete feature space, applies ReLU, and keeps the TopK largest activations\. The decoder reconstructs the original supervision embedding from these sparse activations and normalizes its output\. In main experiments, the sparse autoencoder uses 131072 features with TopK set to 64\. We train it with AdamW using a learning rate of1×10−41\\times 10^\{\-4\}, zero weight decay, a batch size of 128, and 100 epochs\. We use 5% of data for validation and select the best checkpoint based on validation reconstruction loss\.

#### Coarse to Fine Selection\.

The external quality score is obtained by feeding each sample into a multimodal model\. For each sample, we use a discrete rating scale from 1 to 5, and normalize the rating to\[0,1\]\[0,1\]\. Specifically, the scores1,2,3,4,51,2,3,4,5are mapped to0,0\.25,0\.5,0\.75,10,0\.25,0\.5,0\.75,1, respectively\. The prompts used for quality scoring are shown in Figure[8](https://arxiv.org/html/2608.16927#A6.F8)and Figure[9](https://arxiv.org/html/2608.16927#A6.F9)\.

## Appendix CDatasets and Preprocessing Details

### C\.1Datasets

#### Vision\-Flan\.

Vision\-Flan\(Xuet al\.[2024](https://arxiv.org/html/2608.16927#bib.bib29)\)is a high\-quality human\-annotated visual instruction tuning dataset designed to improve the generalization ability of vision\-language models across diverse visual tasks\. It is constructed from publicly available academic vision datasets and covers 187 fine\-grained visual tasks, each paired with expert\-written and carefully validated task instructions\. The dataset reduces synthetic bias and hallucination risk while providing broad task coverage, containing approximately 1000 samples per task and about 186K samples in total\.

#### LLaVA\-CoT\.

LLaVA\-CoT\-100k\(Xuet al\.[2025](https://arxiv.org/html/2608.16927#bib.bib30)\)is an image\-text reasoning instruction dataset designed to improve the structured reasoning ability of vision\-language models\. It integrates approximately 99K image\-question\-answer pairs from a mixture of general\-purpose and science\-targeted VQA datasets\. Each sample is annotated with a structured reasoning response consisting of summary, caption, reasoning, and conclusion stages\.

### C\.2Preprocessing Procedures

For Vision\-Flan, we use the original dataset without additional filtering or format conversion\. For LLaVA\-CoT, we remove samples without associated images, resulting in 98572 image\-text reasoning samples\. The reasoning process and final answer in each response are wrapped with`<think\></think\>`and`<answer\></answer\>`, respectively\.

## Appendix DTraining Hyperparameters

Table[5](https://arxiv.org/html/2608.16927#A6.T5)summarizes the training hyperparameters used for all target models\. For LLaVA\-V1\.5\-7B and LLaVA\-V1\.5\-13B, we follow the hyperparameter settings from the official LLaVA training code\. For Qwen2\-VL\-2B\-Instruct and Llama\-3\.2\-11B\-Vision\-Instruct, since both models are already instruction\-tuned checkpoints, we use a conservative learning rate of1×10−51\\times 10^\{\-5\}for full\-parameter fine\-tuning\. We set the batch size to 64 for these two models because using a batch size of 128 leads to too few optimization steps within one epoch, both for the full dataset and for subset training, making the training insufficient\.

## Appendix EEvaluation

### E\.1Evaluation Benchmarks

We choose benchmarks according to the modality and target capability of each dataset\. For Vision\-Flan, we use benchmarks covering general multimodal understanding, visual question answering, OCR, document understanding, chart understanding, and science\-diagram reasoning\. For LLaVA\-CoT, we combine multimodal mathematical and logical reasoning benchmarks with general vision\-language benchmarks to evaluate both reasoning ability and overall multimodal robustness\. For Qwen2\-VL\-2B\-Instruct trained on LLaVA\-CoT, we observe that full\-data training still yields extremely poor performance on We\-Math\(Qiaoet al\.[2025](https://arxiv.org/html/2608.16927#bib.bib55)\), LogicVista\(Xiaoet al\.[2024](https://arxiv.org/html/2608.16927#bib.bib56)\), DynaMath\(Zouet al\.[2024](https://arxiv.org/html/2608.16927#bib.bib57)\), and MMStar\(Chenet al\.[2024](https://arxiv.org/html/2608.16927#bib.bib52)\), making these benchmarks less informative for comparing data selection methods under this target model\. Therefore, for this model we evaluate on the remaining eight benchmarks\. The full benchmark list is shown in Table[6](https://arxiv.org/html/2608.16927#A6.T6)\.

### E\.2Detailed Evaluation Results

For the main experiments, Tables[7](https://arxiv.org/html/2608.16927#A6.T7)and[8](https://arxiv.org/html/2608.16927#A6.T8)present the full benchmark\-level comparison results on Vision\-Flan and LLaVA\-CoT, respectively, covering MASS and all data selection baselines\. We further provide benchmark\-level results for the analysis and ablation studies discussed in the main paper and appendix\. Specifically, Table[9](https://arxiv.org/html/2608.16927#A6.T9)reports the ablation results of DAE manifold coordinates\. Table[10](https://arxiv.org/html/2608.16927#A6.T10)analyzes the complementarity between DAE and SAE\. Table[11](https://arxiv.org/html/2608.16927#A6.T11)reports the sensitivity analysis of the DAE latent dimension\. Table[12](https://arxiv.org/html/2608.16927#A6.T12)reports the sensitivity analysis of the SAE feature dimension and TopK sparsity\. Table[13](https://arxiv.org/html/2608.16927#A6.T13)reports the results under different quality signal sources\. Table[14](https://arxiv.org/html/2608.16927#A6.T14)reports the results under different embedding sources\. Finally, Table[15](https://arxiv.org/html/2608.16927#A6.T15)shows the Vision\-Flan results with LLaVA\-V1\.5\-13B as the target model, while Table[16](https://arxiv.org/html/2608.16927#A6.T16)shows the LLaVA\-CoT results with Qwen2\-VL\-2B\-Instruct\.

## Appendix FTime Cost Analysis

We compare the time cost of different data selection methods on LLaVA\-CoT\. Table[17](https://arxiv.org/html/2608.16927#A6.T17)reports the GPU hours required by each method on a single NVIDIA A6000\. MASS takes approximately 15\.39 GPU hours in total\. Although its computational cost is slightly higher than that of some baselines, most of the time is spent on quality scoring and embedding extraction, and these results can be reused in other data usage scenarios, such as subsequent curriculum learning\. In addition, compared with the cost of full SFT training, this preprocessing overhead remains acceptable\. As shown by the main experimental results, MASS achieves the best performance under multiple data budgets, demonstrating a favorable balance between selection effectiveness and computational cost\.

MetricVision\-FlanLLaVA\-CoTEmbedding dimension20482048PCA components for 90% variance379459PCA effective rank255\.06275\.17TwoNN intrinsic dimension8\.2511\.88Local PCA dimension \(mean\)17\.3820\.88Local PCA dimension \(median\)1921PCA neighborhood preservation0\.3830\.281Isomap neighborhood preservation0\.4990\.380Table 4:Embedding geometry statistics of Vision\-Flan and LLaVA\-CoT\. Both datasets exhibit globally complex but locally low\-dimensional nonlinear structures\.Table 5:Training hyperparameters for target models\.Table 6:Evaluation benchmarks used for each dataset\.\(a\)5% data subset\.
\(b\)10% data subset\.
\(c\)15% data subset\.

Table 7:Main experimental results on Vision\-Flan using LLaVA\-v1\.5\-7B as the target model\. The best result in each column is highlighted in bold\.\(a\)5% data subset\.
\(b\)10% data subset\.
\(c\)15% data subset\.

Table 8:Main experimental results on LLaVA\-CoT using Llama\-3\.2\-11B\-Vision\-Instruct as the target model\. The best result in each column is highlighted in bold\.Table 9:Ablation results of DAE manifold coordinates\. We compare MASS with raw embedding clustering and DAE principal manifold clustering on the Vision\-Flan dataset using LLaVA\-v1\.5\-7B with a 10% data subset\. The best result in each column is highlighted in bold\.Table 10:Ablation results of the complementarity of DAE and SAE\. We evaluate random selection, DAE\-only grouping, SAE\-only coverage selection, and the full MASS method on the Vision\-Flan dataset using LLaVA\-v1\.5\-7B with a 10% data subset\. The best result in each column is highlighted in bold\.Table 11:Sensitivity analysis results of DAE latent dimension\. We vary the DAE latent dimension on the Vision\-Flan dataset using LLaVA\-v1\.5\-7B with a 10% data subset, while keeping all other MASS settings unchanged\. The best result in each column is highlighted in bold\.\(a\)Effect of SAE feature dimension under the 5% data subset\.
\(b\)Effect of SAE feature dimension under the 10% data subset\.
\(c\)Effect of SAE feature dimension under the 15% data subset\.
\(d\)Effect of TopK sparsity under the 10% data subset with SAE feature dimension fixed to 128\.

Table 12:Sensitivity analysis results of SAE configuration\. We analyze the effect of SAE feature dimension under 5%, 10%, and 15% data budgets, and the effect of TopK sparsity under the 10% data budget on the Vision\-Flan dataset using LLaVA\-v1\.5\-7B\. The best result in each column is highlighted in bold\.Table 13:Ablation results of quality signals\. We evaluate MASS without quality signals and with different external scoring models on the Vision\-Flan dataset using LLaVA\-v1\.5\-7B with a 10% data subset\. The best result in each column is highlighted in bold\.Table 14:Ablation results of the embedding source\. We replace the original embedding source with GME on the Vision\-Flan dataset using LLaVA\-v1\.5\-7B with a 10% data subset, while keeping all other MASS settings unchanged\. The best result in each column is highlighted in bold\.\(a\)5% data subset\.
\(b\)10% data subset\.
\(c\)15% data subset\.

Table 15:Robustness results across target models on Vision\-Flan using LLaVA\-v1\.5\-13B as the target model\. The best result in each column is highlighted in bold\.\(a\)5% data subset\.
\(b\)10% data subset\.
\(c\)15% data subset\.

Table 16:Robustness results across target models on LLaVA\-CoT using Qwen2VL\-2B\-Instruct as the target model\. The best result in each column is highlighted in bold\.Table 17:Time cost comparison of different methods on LLaVA\-CoT\.![Refer to caption](https://arxiv.org/html/2608.16927v1/images/prompt_vf_mm.png)Figure 8:Scoring prompt for estimating the quality score on Vision\-Flan\.![Refer to caption](https://arxiv.org/html/2608.16927v1/images/prompt_llavacot.png)Figure 9:Scoring prompt for estimating the quality score on LLaVA\-CoT\.
## References

- A\. Abbas, K\. Tirumala, D\. Simig, S\. Ganguli, and A\. S\. Morcos \(2023\)Semdedup: data\-efficient learning at web\-scale through semantic deduplication\.arXiv preprint arXiv:2303\.09540\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px4.p1.1)\.
- M\. Aharon, M\. Elad, and A\. Bruckstein \(2006\)K\-svd: an algorithm for designing overcomplete dictionaries for sparse representation\.IEEE Transactions on signal processing54\(11\),pp\. 4311–4322\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px3.p1.1)\.
- X\. An, Y\. Xie, K\. Yang, W\. Zhang, X\. Zhao, Z\. Cheng, Y\. Wang, S\. Xu, C\. Chen, D\. Zhu,et al\.\(2025\)Llava\-onevision\-1\.5: fully open framework for democratized multimodal training\.arXiv preprint arXiv:2509\.23661\.Cited by:[§5\.5](https://arxiv.org/html/2608.16927#S5.SS5.p1.1)\.
- S\. Bai, Y\. Cai, R\. Chen, K\. Chen, X\. Chen, Z\. Cheng, L\. Deng, W\. Ding, C\. Gao, C\. Ge,et al\.\(2025\)Qwen3\-vl technical report\.arXiv preprint arXiv:2511\.21631\.Cited by:[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px5.p1.1)\.
- P\. Baldi and K\. Hornik \(1989\)Neural networks and principal component analysis: learning from examples without local minima\.Neural networks2\(1\),pp\. 53–58\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Bengio, A\. Courville, and P\. Vincent \(2013\)Representation learning: a review and new perspectives\.IEEE transactions on pattern analysis and machine intelligence35\(8\),pp\. 1798–1828\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Bi, Y\. Wang, D\. Yan, W\. Huang, Z\. Jin, X\. Ma, S\. Yan, A\. Hecker, M\. Ye, X\. Xiao,et al\.\(2025\)Prism: self\-pruning intrinsic selection method for training\-free multimodal data selection\.arXiv preprint arXiv:2502\.12119\.Cited by:[§1](https://arxiv.org/html/2608.16927#S1.p1.1),[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px4.p1.1)\.
- T\. Bricken, A\. Templeton, J\. Batson, B\. Chen, A\. Jermyn, T\. Conerly, N\. Turner, C\. Anil, C\. Denison, A\. Askell,et al\.\(2023\)Towards monosemanticity: decomposing language models with dictionary learning\.Transformer Circuits Thread2\(5\),pp\. 6\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px3.p1.1)\.
- L\. Chen, J\. Li, X\. Dong, P\. Zhang, Y\. Zang, Z\. Chen, H\. Duan, J\. Wang, Y\. Qiao, D\. Lin,et al\.\(2024\)Are we on the right way for evaluating large vision\-language models?\.Advances in Neural Information Processing Systems37,pp\. 27056–27087\.Cited by:[§E\.1](https://arxiv.org/html/2608.16927#A5.SS1.p1.1),[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.12.11.1),[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.18.17.1)\.
- Y\. Chen, Y\. Li, K\. Hu, M\. Zerun, H\. HaochenYe, and K\. Chen \(2025\)Mig: automatic data selection for instruction tuning by maximizing information gain in semantic space\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 9902–9915\.Cited by:[§1](https://arxiv.org/html/2608.16927#S1.p2.1)\.
- R\. Deb, K\. Thekumparampil, K\. Kalantari, G\. Hiranandani, S\. Sabach, and B\. Kveton \(2025\)Fishersft: data\-efficient supervised fine\-tuning of language models using information gain\.arXiv preprint arXiv:2505\.14826\.Cited by:[§1](https://arxiv.org/html/2608.16927#S1.p1.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px2.p1.1)\.
- D\. Gurari, Q\. Li, A\. J\. Stangl, A\. Guo, C\. Lin, K\. Grauman, J\. Luo, and J\. P\. Bigham \(2018\)Vizwiz grand challenge: answering visual questions from blind people\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\. 3608–3617\.Cited by:[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.3.2.1)\.
- J\. Hessel, A\. Holtzman, M\. Forbes, R\. Le Bras, and Y\. Choi \(2021\)Clipscore: a reference\-free evaluation metric for image captioning\.InProceedings of the 2021 conference on empirical methods in natural language processing,pp\. 7514–7528\.Cited by:[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px4.p1.1)\.
- G\. E\. Hinton and R\. R\. Salakhutdinov \(2006\)Reducing the dimensionality of data with neural networks\.science313\(5786\),pp\. 504–507\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px2.p1.1)\.
- R\. Huben, H\. Cunningham, L\. Smith, A\. Ewart, and L\. Sharkey \(2024\)Sparse autoencoders find highly interpretable features in language models\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 7827–7845\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px3.p1.1)\.
- D\. A\. Hudson and C\. D\. Manning \(2019\)Gqa: a new dataset for real\-world visual reasoning and compositional question answering\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 6700–6709\.Cited by:[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.2.1.2)\.
- H\. Ivison, M\. Zhang, F\. Brahman, P\. W\. Koh, and P\. Dasigi \(2025\)Large\-scale data selection for instruction tuning\.arXiv preprint arXiv:2503\.01807\.Cited by:[§1](https://arxiv.org/html/2608.16927#S1.p1.1)\.
- A\. Kembhavi, M\. Salvato, E\. Kolve, M\. Seo, H\. Hajishirzi, and A\. Farhadi \(2016\)A diagram is worth a dozen images\.InEuropean conference on computer vision,pp\. 235–251\.Cited by:[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.22.21.1),[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.8.7.1)\.
- M\. A\. Kramer \(1991\)Nonlinear principal component analysis using autoassociative neural networks\.AIChE journal37\(2\),pp\. 233–243\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px2.p1.1)\.
- W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. Gonzalez, H\. Zhang, and I\. Stoica \(2023\)Efficient memory management for large language model serving with pagedattention\.InProceedings of the 29th symposium on operating systems principles,pp\. 611–626\.Cited by:[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px5.p1.1)\.
- J\. Lee, B\. Li, and S\. J\. Hwang \(2024\)Concept\-skill transferability\-based data selection for large vision\-language models\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 5060–5080\.Cited by:[§1](https://arxiv.org/html/2608.16927#S1.p1.1),[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px4.p1.1)\.
- M\. Li, Y\. Zhang, D\. Long, K\. Chen, S\. Song, S\. Bai, Z\. Yang, P\. Xie, A\. Yang, D\. Liu,et al\.\(2026\)Qwen3\-vl\-embedding and qwen3\-vl\-reranker: a unified framework for state\-of\-the\-art multimodal retrieval and ranking\.arXiv preprint arXiv:2601\.04720\.Cited by:[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px5.p1.1)\.
- Z\. Liang, Y\. Xu, Y\. Hong, P\. Shang, Q\. Wang, Q\. Fu, and K\. Liu \(2024\)A survey of multimodel large language models\.InProceedings of the 3rd International Conference on Computer, Artificial Intelligence and Control Engineering,pp\. 405–409\.Cited by:[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.19.18.1),[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.6.5.1)\.
- H\. Liu, C\. Li, Y\. Li, and Y\. J\. Lee \(2024a\)Improved baselines with visual instruction tuning\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 26296–26306\.Cited by:[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px2.p1.1),[§5\.7](https://arxiv.org/html/2608.16927#S5.SS7.p1.1)\.
- Y\. Liu, H\. Duan, Y\. Zhang, B\. Li, S\. Zhang, W\. Zhao, Y\. Yuan, J\. Wang, C\. He, Z\. Liu,et al\.\(2024b\)Mmbench: is your multi\-modal model an all\-around player?\.InEuropean conference on computer vision,pp\. 216–233\.Cited by:[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.20.19.1),[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.7.6.1)\.
- Y\. Liu, Z\. Li, M\. Huang, B\. Yang, W\. Yu, C\. Li, X\. Yin, C\. Liu, L\. Jin, and X\. Bai \(2024c\)Ocrbench: on the hidden mystery of ocr in large multimodal models\.Science China Information Sciences67\(12\),pp\. 220102\.Cited by:[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.13.12.1),[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.25.24.1)\.
- Z\. Liu, K\. Zhou, W\. X\. Zhao, D\. Gao, Y\. Li, and J\. Wen \(2025\)Less is more: high\-value data selection for visual instruction tuning\.InProceedings of the 33rd ACM International Conference on Multimedia,pp\. 3712–3721\.Cited by:[§1](https://arxiv.org/html/2608.16927#S1.p1.1)\.
- K\. Lu, H\. Yuan, Z\. Yuan, R\. Lin, J\. Lin, C\. Tan, C\. Zhou, and J\. Zhou \(2023\)\# instag: instruction tagging for analyzing supervised fine\-tuning of large language models\.arXiv preprint arXiv:2308\.07074\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Maharana, P\. Yadav, and M\. Bansal \(2023\)D2 pruning: message passing for balancing diversity and difficulty in data pruning\.arXiv preprint arXiv:2310\.07931\.Cited by:[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px4.p1.1)\.
- A\. Makhzani and B\. Frey \(2013\)K\-sparse autoencoders\.arXiv preprint arXiv:1312\.5663\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px3.p1.1)\.
- A\. Masry, X\. L\. Do, J\. Q\. Tan, S\. Joty, and E\. Hoque \(2022\)Chartqa: a benchmark for question answering about charts with visual and logical reasoning\.InFindings of the association for computational linguistics: ACL 2022,pp\. 2263–2279\.Cited by:[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.23.22.1),[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.9.8.1)\.
- M\. Mathew, V\. Bagal, R\. Tito, D\. Karatzas, E\. Valveny, and C\. Jawahar \(2022\)Infographicvqa\.InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision,pp\. 1697–1706\.Cited by:[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.11.10.1),[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.24.23.1)\.
- M\. Mathew, D\. Karatzas, and C\. Jawahar \(2021\)Docvqa: a dataset for vqa on document images\.InProceedings of the IEEE/CVF winter conference on applications of computer vision,pp\. 2200–2209\.Cited by:[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.10.9.1)\.
- N\. Naharas, D\. Nguyen, N\. Bulut, M\. Bateni, V\. Mirrokni, and B\. Mirzasoleiman \(2025\)Data selection for fine\-tuning vision language models via cross modal alignment trajectories\.arXiv preprint arXiv:2510\.01454\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px4.p1.1)\.
- B\. A\. Olshausen and D\. J\. Field \(1997\)Sparse coding with an overcomplete basis set: a strategy employed by v1?\.Vision research37\(23\),pp\. 3311–3325\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px3.p1.1)\.
- M\. Paul, S\. Ganguli, and G\. K\. Dziugaite \(2021\)Deep learning on a data diet: finding important examples early in training\.Advances in neural information processing systems34,pp\. 20596–20607\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px4.p1.1)\.
- R\. Qiao, Q\. Tan, G\. Dong, M\. MinhuiWu, C\. Sun, X\. Song, J\. Wang, Z\. Gongque, S\. Lei, Y\. Zhang,et al\.\(2025\)We\-math: does your large multimodal model achieve human\-like mathematical reasoning?\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 20023–20070\.Cited by:[§E\.1](https://arxiv.org/html/2608.16927#A5.SS1.p1.1),[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.15.14.1)\.
- S\. Rajamanoharan, A\. Conmy, L\. Smith, T\. Lieberum, V\. Varma, J\. Kramár, R\. Shah, and N\. Nanda \(2024a\)Improving dictionary learning with gated sparse autoencoders\.arXiv preprint arXiv:2404\.16014\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px3.p1.1)\.
- S\. Rajamanoharan, T\. Lieberum, N\. Sonnerat, A\. Conmy, V\. Varma, J\. Kramár, and N\. Nanda \(2024b\)Jumping ahead: improving reconstruction fidelity with jumprelu sparse autoencoders\.arXiv preprint arXiv:2407\.14435\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px3.p1.1)\.
- T\. Saikh, T\. Ghosal, A\. Mittal, A\. Ekbal, and P\. Bhattacharyya \(2022\)Scienceqa: a novel resource for question answering on scholarly articles\.International Journal on Digital Libraries23\(3\),pp\. 289–301\.Cited by:[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.21.20.1),[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.5.4.1)\.
- A\. Singh, V\. Natarajan, M\. Shah, Y\. Jiang, X\. Chen, D\. Batra, D\. Parikh, and M\. Rohrbach \(2019\)Towards vqa models that can read\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 8317–8326\.Cited by:[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.4.3.1)\.
- P\. Sun, Y\. Yang, H\. Shen, Y\. Ban, T\. Fu, Y\. Wang, and Y\. Li \(2026\)Does the question really matter? training\-free data selection for vision\-language sft\.arXiv preprint arXiv:2603\.09715\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px1.p1.1)\.
- Q\. Team \(2026\)Qwen3\. 5\-omni technical report\.arXiv preprint arXiv:2604\.15804\.Cited by:[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px5.p1.1)\.
- K\. Wang, J\. Pan, W\. Shi, Z\. Lu, H\. Ren, A\. Zhou, M\. Zhan, and H\. Li \(2024a\)Measuring multimodal mathematical reasoning with math\-vision dataset\.Advances in Neural Information Processing Systems37,pp\. 95095–95169\.Cited by:[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.14.13.2)\.
- P\. Wang, S\. Bai, S\. Tan, S\. Wang, Z\. Fan, J\. Bai, K\. Chen, X\. Liu, J\. Wang, W\. Ge,et al\.\(2024b\)Qwen2\-vl: enhancing vision\-language model’s perception of the world at any resolution\.arXiv preprint arXiv:2409\.12191\.Cited by:[§5\.7](https://arxiv.org/html/2608.16927#S5.SS7.p1.1)\.
- S\. Wang, X\. Ouyang, T\. Xu, Y\. Hu, J\. Liu, G\. Chen, T\. Zhang, J\. Zheng, K\. Yang, X\. Ren,et al\.\(2026\)OPUS: towards efficient and principled data selection in large language model pre\-training in every iteration\.arXiv preprint arXiv:2602\.05400\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px1.p1.1)\.
- C\. Wu, J\. Mao, Y\. Miao, S\. Lian, B\. Yu, X\. Lin, C\. Huang, L\. Zhang, and K\. Chen \(2026\)ScalSelect: scalable training\-free multimodal data selection for efficient visual instruction tuning\.arXiv preprint arXiv:2602\.11636\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px4.p1.1)\.
- X\. Wu, M\. Xia, R\. Shao, Z\. Deng, P\. W\. Koh, and O\. Russakovsky \(2024\)Icons: influence consensus for vision\-language data selection\.arXiv preprint arXiv:2501\.00654\.Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px1.p1.1)\.
- M\. Xia, S\. Malladi, S\. Gururangan, S\. Arora, and D\. Chen \(2024\)LESS: selecting influential data for targeted instruction tuning\.External Links:2402\.04333,[Link](https://arxiv.org/abs/2402.04333)Cited by:[§2](https://arxiv.org/html/2608.16927#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Xiao, E\. Sun, T\. Liu, and W\. Wang \(2024\)Logicvista: multimodal llm logical reasoning benchmark in visual contexts\.arXiv preprint arXiv:2407\.04973\.Cited by:[§E\.1](https://arxiv.org/html/2608.16927#A5.SS1.p1.1),[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.16.15.1)\.
- G\. Xu, P\. Jin, Z\. Wu, H\. Li, Y\. Song, L\. Sun, and L\. Yuan \(2025\)Llava\-cot: let vision language models reason step\-by\-step\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 2087–2098\.Cited by:[§C\.1](https://arxiv.org/html/2608.16927#A3.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px1.p1.1)\.
- Z\. Xu, C\. Feng, R\. Shao, T\. Ashby, Y\. Shen, D\. Jin, Y\. Cheng, Q\. Wang, and L\. Huang \(2024\)Vision\-flan: scaling human\-labeled tasks in visual instruction tuning\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 15271–15342\.Cited by:[§C\.1](https://arxiv.org/html/2608.16927#A3.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.16927#S4.SS1.SSS0.Px1.p1.1)\.
- Y\. Yan, M\. Zhong, Q\. Zhu, X\. Gu, J\. Chen, and H\. Li \(2025\)CoIDO: efficient data selection for visual instruction tuning via coupled importance\-diversity optimization\.arXiv preprint arXiv:2510\.17847\.Cited by:[§1](https://arxiv.org/html/2608.16927#S1.p1.1)\.
- X\. Zhang, Y\. Zhang, W\. Xie, M\. Li, Z\. Dai, D\. Long, P\. Xie, M\. Zhang, W\. Li, and M\. Zhang \(2024\)GME: improving universal multimodal retrieval by multimodal llms\.arXiv preprint arXiv:2412\.16855\.Cited by:[§5\.6](https://arxiv.org/html/2608.16927#S5.SS6.p1.1)\.
- C\. Zou, X\. Guo, R\. Yang, J\. Zhang, B\. Hu, and H\. Zhang \(2024\)Dynamath: a dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models\.arXiv preprint arXiv:2411\.00836\.Cited by:[§E\.1](https://arxiv.org/html/2608.16927#A5.SS1.p1.1),[Table 6](https://arxiv.org/html/2608.16927#A6.T6.1.17.16.1)\.