DataComp-VLM: Improved Open Datasets for Vision-Language Models
Summary
This paper introduces DataComp-VLM (DCVLM), a comprehensive benchmark for evaluating data curation strategies for vision-language models. The authors find that data mixing, rather than filtering, significantly improves performance, and their resulting DCVLM-Baseline dataset achieves state-of-the-art results on 33 downstream tasks.
View Cached Full Text
Cached at: 07/06/26, 10:36 AM
Paper page - DataComp-VLM: Improved Open Datasets for Vision-Language Models
Source: https://huggingface.co/papers/2606.28551 Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
DataComp for VLMs (DCVLM) establishes a comprehensive benchmark for evaluating data curation strategies in vision-language models, demonstrating that data mixing rather than filtering significantly improves model performance at scale.
Building performantVision-Language Models(VLMs) requires carefully curating large-scale training datasets, yet the community lacks systematic benchmarks for evaluating such curation strategies. We introduce DataComp for VLMs (DCVLM), a benchmark for controlled data-centric experiments to improve VLM training. As part of DCVLM, we collect 160 datasets spanning four data types -- image-caption pairs, multimodal interleaved documents, text-only, and instruction-tuning data -- into a corpus of 6Tmultimodal tokens. DCVLM allows participants to test curation strategies (filtering, mixing, formatting, sampling) across 1B-8B models and 6.25B-200B token budgets. Models are then evaluated on a carefully selected suite of up to 52downstream benchmarksacross 9 domains. We conduct extensive experiments on DCVLM and find thatdata mixing, not filtering, is key to a high-quality training dataset: instruction-heavy mixtures scale better than caption-heavy ones, with gains widening at larger scales. The resulting dataset, DCVLM-Baseline, enables training an 8B VLM to 63.6% accuracy on our 33-task core suite with 200B training tokens. Compared to FineVision, the state-of-the-art open VLM training dataset, this represents an improvement of +5.4pp. DCVLM and all accompanying artifacts will be made publicly available at https://www.datacomp.ai/dcvlm/.
View arXiv pageView PDFProject pageGitHub23Add to collection
Get this paper in your agent:
hf papers read 2606\.28551
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.28551 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.28551 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.28551 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
CaVe-VLM-CoT: An Interpretable Vision-Language Model Framework
CaVe-VLM-CoT is a modular reflection-based agentic-RAG framework for vision-language models that enforces evidence-grounded reasoning through a five-stage pipeline, achieving 87.1% accuracy on ScienceQA and proposing a suite of 23 metrics for evaluation.
VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback?
Introduces VisEditBench, a benchmark of 1,395 human-annotated visualization code-editing tasks for vision-language models, covering feedback-guided repair and reference-guided restyling. Evaluates 20 VLMs and proposes VisEditAgent, a render-grounded editing framework that improves pass rates from 55.75% to 67.99%.
Large Language Model Teaches Visual Students: Cross-Modality Transfer of Fine-Grained Conceptual Knowledge
This paper introduces LaViD, a framework that transfers semantic knowledge from a language-only LLM to a vision student model by generating multiple-choice questions as conceptual signatures, achieving superior fine-grained classification performance and robustness.
HyperGVL: Benchmarking and Improving Large Vision-Language Models in Hypergraph Understanding and Reasoning
HyperGVL introduces the first benchmark for evaluating Large Vision-Language Models on hypergraph understanding and reasoning, featuring 84,000 QA samples across 12 tasks and real-world applications. The paper also proposes WiseHyGR, a generalizable router that enhances LVLM performance through adaptive hypergraph representations.
AnyGroundBench: A Specialized-Domain Benchmark for Video Grounding in Vision-Language Models
Introduces AnyGroundBench, a domain-adaptation benchmark for spatio-temporal video grounding, evaluating 15 VLMs across five specialized domains and finding current models fail in zero-shot and in-context learning adaptation.