World Embedding Benchmark

Hugging Face Daily Papers Papers
world-models video-generation embeddings physical-reasoning benchmark multimodal

Summary

This paper introduces the World Embedding Benchmark, with 8,000 controlled simulation cases spanning fluid/solid mechanics, dynamics, and optics, to evaluate how video representations encode physical information. It finds pretrained multimodal embeddings struggle with physical alignment, probes recover physics from frozen embeddings, contrastive training trades off alignment vs. recoverability, and retrieved reference videos improve physical fidelity of MiniMax-H3-generated videos.

Physical fidelity has received increasing attention in world models and video generation, yet how video representations encode physical information remains less understood. We introduce the World Embedding Benchmark, comprising 8,000 controlled simulation cases from 80 families spanning fluid mechanics, solid mechanics, dynamics, and optics & electromagnetism. Each case pairs a rendered video with simulation-derived physical annotations, supporting three complementary tasks: text-video retrieval, physical-property regression, and multiple-choice video-description pair classification. We use these tasks to distinguish cross-modal physical alignment from the recoverability of quantitative physical information. Evaluated pre-trained omnimodal embedding models show weak retrieval and near-chance within-family pair classification, while lightweight probes recover useful physical information from frozen video embeddings. Continual contrastive training with physics-specific video-text pairs improves retrieval and pair classification but degrades physical-property regression, revealing a trade-off between alignment and quantitative information recoverability. Finally, we use the embeddings to retrieve reference videos for retrieval-augmented generation with MiniMax-H3. Retrieved references improve the physical fidelity of generated videos, with stronger retrieval models yielding larger gains in our experiments. Together, these findings highlight the need to evaluate physical alignment and property recoverability jointly, and demonstrate the utility of physical representations for improving video generation.
Original Article
View Cached Full Text

Cached at: 10/05/26, 04:46 AM

Paper page - World Embedding Benchmark

Source: https://huggingface.co/papers/2610.03632

Abstract

Physicalfidelityhasreceivedincreasingattentioninworldmodelsandvideogeneration,yethowvideorepresentationsencodephysicalinformationremainslessunderstood.WeintroducetheWorldEmbeddingBenchmark,comprising8,000controlledsimulationcasesfrom80familiesspanningfluidmechanics,solidmechanics,dynamics,andoptics&electromagnetism.Eachcasepairsarenderedvideowithsimulation-derivedphysicalannotations,supportingthreecomplementarytasks:text-videoretrieval,physical-propertyregression,andmultiple-choicevideo-descriptionpairclassification.Weusethesetaskstodistinguishcross-modalphysicalalignmentfromtherecoverabilityofquantitativephysicalinformation.Evaluatedpre-trainedomnimodalembeddingmodelsshowweakretrievalandnear-chancewithin-familypairclassification,whilelightweightprobesrecoverusefulphysicalinformationfromfrozenvideoembeddings.Continualcontrastivetrainingwithphysics-specificvideo-textpairsimprovesretrievalandpairclassificationbutdegradesphysical-propertyregression,revealingatrade-offbetweenalignmentandquantitativeinformationrecoverability.Finally,weusetheembeddingstoretrievereferencevideosforretrieval-augmentedgenerationwithMiniMax-H3.Retrievedreferencesimprovethephysicalfidelityofgeneratedvideos,withstrongerretrievalmodelsyieldinglargergainsinourexperiments.Together,thesefindingshighlighttheneedtoevaluatephysicalalignmentandpropertyrecoverabilityjointly,anddemonstratetheutilityofphysicalrepresentationsforimprovingvideogeneration.

View arXiv pageView PDFGitHub1Add to collection

Get this paper in your agent:

hf papers read 2610\.03632

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.03632 in a model README.md to link it from this page.

Datasets citing this paper5

#### World-Representation-Lab/World-Embedding-Fluid-Retrieval Viewer• Updatedabout 2 hours ago • 700 #### World-Representation-Lab/World-Embedding-Solid-Retrieval Viewer• Updatedabout 2 hours ago • 2.7k #### World-Representation-Lab/World-Embedding-Dynamics-Retrieval Viewer• Updatedabout 2 hours ago • 1.9k #### World-Representation-Lab/World-Embedding-Optics-Retrieval Viewer• Updatedabout 2 hours ago • 2.7k Browse 5 datasets citing this paper### Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.03632 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

MVEB: Massive Video Embedding Benchmark

Hugging Face Daily Papers

This paper introduces MVEB, a large-scale benchmark for evaluating video embeddings across 23 tasks, finding that no single model dominates and that audio's contribution depends on dataset annotation provenance. It integrates into the MTEB ecosystem for unified multimodal evaluation.

tencent/WeMM-Embedding 9B/4B/2B

Reddit r/LocalLLaMA

Tencent introduces WeMM-Embedding, a series of universal multimodal embedding models in 9B, 4B, and 2B sizes, built on Qwen3.5, supporting text, images, videos, and visual documents for embedding generation.