OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation

Hugging Face Daily Papers Papers
benchmark dataset video-generation reference-to-video evaluation ai-research

Summary

The paper introduces OmniVBench, a comprehensive benchmark for omni reference-to-video generation, and the Omni-R2VDataset, a large-scale training dataset, to evaluate and improve R2V models.

Reference-to-video (R2V) generation is evolving toward increasingly general and versatile reference control, giving rise to the emerging paradigm of omni R2V generation. However, existing benchmarks fall short of these emerging capabilities: their test cases cover limited reference types and compositions, and their evaluation protocols largely assess holistic reference consistency, overlooking whether reference factors are properly preserved, disentangled, and routed. Meanwhile, the high cost of constructing omni R2V training data makes suitable training resources scarce. To address these gaps, we introduce OmniVBench and the Omni-R2V Dataset for evaluating and training omni R2V models. OmniVBench expands R2V evaluation across broader reference types, fine-grained control tasks, and richer reference compositions, covering 7 task families and 18 fine-grained tasks spanning content, motion, style, structure, narrative, and multi-reference settings. We introduce factor-grounded evaluation with 12,172 case-specific checklist items, assessing whether intended reference factors are faithfully preserved, correctly disentangled and bound to their targets, and properly realized according to the instruction. We further introduce the Omni-R2V Dataset, bringing industrial-grade training resources for diverse R2V tasks to the broader research community. Drawing primarily on a large-scale corpus of professional video footage, it comprises 340K processed training samples spanning diverse reference types and multi-reference compositions. We develop task-specific pipelines for reference-target pair construction, offering a practical and scalable recipe for omni R2V data construction. Extensive evaluation of advanced open- and closed-source R2V models reveals clear performance gaps across task families and evaluation dimensions on OmniVBench, highlighting remaining limitations of current R2V models.
Original Article
View Cached Full Text

Cached at: 09/21/26, 03:20 AM

Paper page - OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation

Source: https://huggingface.co/papers/2609.22069 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Reference-to-video(R2V)generationisevolvingtowardincreasinglygeneralandversatilereferencecontrol,givingrisetotheemergingparadigmofomniR2Vgeneration.However,existingbenchmarksfallshortoftheseemergingcapabilities:theirtestcasescoverlimitedreferencetypesandcompositions,andtheirevaluationprotocolslargelyassessholisticreferenceconsistency,overlookingwhetherreferencefactorsareproperlypreserved,disentangled,androuted.Meanwhile,thehighcostofconstructingomniR2Vtrainingdatamakessuitabletrainingresourcesscarce.Toaddressthesegaps,weintroduceOmniVBenchandtheOmni-R2VDatasetforevaluatingandtrainingomniR2Vmodels.OmniVBenchexpandsR2Vevaluationacrossbroaderreferencetypes,fine-grainedcontroltasks,andricherreferencecompositions,covering7taskfamiliesand18fine-grainedtasksspanningcontent,motion,style,structure,narrative,andmulti-referencesettings.Weintroducefactor-groundedevaluationwith12,172case-specificchecklistitems,assessingwhetherintendedreferencefactorsarefaithfullypreserved,correctlydisentangledandboundtotheirtargets,andproperlyrealizedaccordingtotheinstruction.WefurtherintroducetheOmni-R2VDataset,bringingindustrial-gradetrainingresourcesfordiverseR2Vtaskstothebroaderresearchcommunity.Drawingprimarilyonalarge-scalecorpusofprofessionalvideofootage,itcomprises340Kprocessedtrainingsamplesspanningdiversereferencetypesandmulti-referencecompositions.Wedeveloptask-specificpipelinesforreference-targetpairconstruction,offeringapracticalandscalablerecipeforomniR2Vdataconstruction.Extensiveevaluationofadvancedopen-andclosed-sourceR2VmodelsrevealsclearperformancegapsacrosstaskfamiliesandevaluationdimensionsonOmniVBench,highlightingremaininglimitationsofcurrentR2Vmodels.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2609\.22069

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.22069 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.22069 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.22069 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

A Very Big Video Reasoning Suite

Papers with Code Trending

This paper introduces the Very Big Video Reasoning (VBVR) dataset and benchmark, a large-scale resource with over one million video clips across 200 reasoning tasks, enabling systematic study of spatiotemporal reasoning and showing early signs of emergent generalization.