Video Generation Models are General-Purpose Vision Learners

Hugging Face Daily Papers Papers
video-generation vision foundation-model generalist pretraining diffusion perception

Summary

This paper proposes that large-scale text-to-video generation can serve as a powerful pre-training paradigm for computer vision, introducing GenCeption which achieves state-of-the-art performance across diverse vision tasks with high data efficiency and emergent generalization to unseen domains.

Driven by next-token prediction, NLP shifted from task-specific models into powerful generalist foundation models. What, then, is the equivalent catalyst needed to achieve a general-purpose model in computer vision? In this paper, we contend that large-scale text-to-video generation serves as a strong pre-training paradigm for computer vision, providing the necessary spatiotemporal priors, vision-language alignment, and scalability required for general visual intelligence. We introduce GenCeption, which leverages a pre-trained video generative diffusion backbone to define a feed-forward perception model, capable of performing various vision tasks steered by text instructions. Empirical results demonstrate that GenCeption achieves state-of-the-art performance across a diverse suite of tasks, including depth, surface normal, and camera pose estimation, expression-referring segmentation, and 3D keypoint prediction, often matching or surpassing specialized models (e.g. DepthAnything3, SAM3, D4RT, VGGT-Omega, Sapiens, David, Genmo, and Lotus-2). Furthermore, the video generative pretrained backbone outperforms alternative pretraining paradigms (e.g., V-JEPA, and Video MAE) under comparable settings. Importantly, GenCeption exhibits preliminary data and model scaling properties along with exceptional data efficiency, where it achieves comparable performance with leading models like D4RT and VGGT-Omega with 7 to 500 less training data. Finally, GenCeption also exhibits intriguing emergent behaviors: a model trained exclusively on synthetic human videos generalizes to real-world footage and out-of-distribution object categories (e.g., animals and robots). These findings suggest that video generation is not merely a synthesis tool, but a foundational path toward generalist vision intelligence for the physical world. Project page: https://genception.github.io
Original Article
View Cached Full Text

Cached at: 07/13/26, 07:50 AM

Paper page - Video Generation Models are General-Purpose Vision Learners

Source: https://huggingface.co/papers/2607.09024 Published on Jul 10

#2 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

Drivenbynext-tokenprediction,NLPshiftedfromtask-specificmodelsintopowerfulgeneralistfoundationmodels.What,then,istheequivalentcatalystneededtoachieveageneral-purposemodelincomputervision?Inthispaper,wecontendthatlarge-scaletext-to-videogenerationservesasastrongpre-trainingparadigmforcomputervision,providingthenecessaryspatiotemporalpriors,vision-languagealignment,andscalabilityrequiredforgeneralvisualintelligence.WeintroduceGenCeption,whichleveragesapre-trainedvideogenerativediffusionbackbonetodefineafeed-forwardperceptionmodel,capableofperformingvariousvisiontaskssteeredbytextinstructions.EmpiricalresultsdemonstratethatGenCeptionachievesstate-of-the-artperformanceacrossadiversesuiteoftasks,includingdepth,surfacenormal,andcameraposeestimation,expression-referringsegmentation,and3Dkeypointprediction,oftenmatchingorsurpassingspecializedmodels(e.g.DepthAnything3,SAM3,D4RT,VGGT-Omega,Sapiens,David,Genmo,andLotus-2).Furthermore,thevideogenerativepretrainedbackboneoutperformsalternativepretrainingparadigms(e.g.,V-JEPA,andVideoMAE)undercomparablesettings.Importantly,GenCeptionexhibitspreliminarydataandmodelscalingpropertiesalongwithexceptionaldataefficiency,whereitachievescomparableperformancewithleadingmodelslikeD4RTandVGGT-Omegawith7to500lesstrainingdata.Finally,GenCeptionalsoexhibitsintriguingemergentbehaviors:amodeltrainedexclusivelyonsynthetichumanvideosgeneralizestoreal-worldfootageandout-of-distributionobjectcategories(e.g.,animalsandrobots).Thesefindingssuggestthatvideogenerationisnotmerelyasynthesistool,butafoundationalpathtowardgeneralistvisionintelligenceforthephysicalworld.Projectpage:https://genception.github.io

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2607\.09024

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.09024 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.09024 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.09024 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

Video Generators as General-Purpose Vision Models (8 minute read)

TLDR AI

GenCeption repurposes pre-trained video generative models into a single unified feed-forward vision model that achieves state-of-the-art performance across multiple tasks with exceptional data efficiency, marking a shift toward general-purpose visual intelligence.

Getting video models to learn better, faster

Hacker News Top

The article explores the evolution of data filtering techniques for generative video pre-training, emphasizing methods like computer vision, LLMs, and reinforcement learning to improve model performance through high-quality data.

Video generation models as world simulators

OpenAI Blog

OpenAI's technical report on Sora describes a video generation model that unifies diverse visual data through visual patches, enabling large-scale training of generative models capable of producing high-definition videos up to one minute long across variable durations, aspect ratios, and resolutions.

Which Pretraining Paradigm Better Serves Spatial Intelligence? An Empirical Comparison of Vision-Language and Video Generation Models

Hugging Face Daily Papers

This paper presents a systematic frozen-feature probing study comparing vision-language models (VLMs) and video generation models (VGMs) on spatial intelligence tasks. It finds that VLMs excel at semantic tagging and instance grouping, while VGMs provide better dense geometry and camera motion signals, and a naive fusion of both yields strong performance across all axes.

Vision as Unified Multimodal Generation

Hugging Face Daily Papers

This paper presents SenseNova-Vision, a unified multimodal model that formulates computer vision tasks as generation problems, achieving performance comparable to specialized systems across diverse vision tasks. It introduces a large-scale instruction-response corpus and publicly releases the model and datasets.