ROWBench: Do Video Models Render What the Program Specifies?

Hugging Face Daily Papers Papers
video-generation world-models benchmark evaluation vlm simulation

Summary

PROWBench introduces 170 programmatically constructed episodes and 600 proxy videos to evaluate whether generative video models faithfully render program-specified world dynamics, using VLM-based metrics for logic-render alignment and interaction success.

Programmable world models separate executable dynamics from visual generation, offering a promising foundation for next-generation game engines. However, their visual adherence to explicit rules and interactions remains insufficiently evaluated. Existing benchmarks assess visual quality, controllability, and instruction or physical adherence, but rarely test fidelity to fine-grained, program-specified world events. We introduce PROWBench, comprising 170 programmatically constructed episodes and 600 proxy videos covering diverse scenes and interactions. PROWBench logs entity states and timestamped events, including those outside the camera's field of view, as replayable world records, from which it renders synchronized views and proxy representations. This enables generated videos to be checked against the observable consequences of program execution. An extensible framework constructs scenes, controls behaviors, and can render each camera view in different representations, such as coarse 3D, and bounding boxes. The benchmark covers first- and third-person perspectives, with synchronized multi-view observations available for a subset of episodes. Grounded in these records, PROWBench evaluates entity control, long-horizon memory, and, with two VLM-based metrics, Logic-Render Alignment and Interaction Success Rate, adherence to the prescribed timeline and the visual realization of timestamped engine-recorded events.
Original Article
View Cached Full Text

Cached at: 10/02/26, 04:27 AM

Paper page - ROWBench: Do Video Models Render What the Program Specifies?

Source: https://huggingface.co/papers/2610.02205

Abstract

Programmableworldmodelsseparateexecutabledynamicsfromvisualgeneration,offeringapromisingfoundationfornext-generationgameengines.However,theirvisualadherencetoexplicitrulesandinteractionsremainsinsufficientlyevaluated.Existingbenchmarksassessvisualquality,controllability,andinstructionorphysicaladherence,butrarelytestfidelitytofine-grained,program-specifiedworldevents.WeintroducePROWBench,comprising170programmaticallyconstructedepisodesand600proxyvideoscoveringdiversescenesandinteractions.PROWBenchlogsentitystatesandtimestampedevents,includingthoseoutsidethecamera’sfieldofview,asreplayableworldrecords,fromwhichitrenderssynchronizedviewsandproxyrepresentations.Thisenablesgeneratedvideostobecheckedagainsttheobservableconsequencesofprogramexecution.Anextensibleframeworkconstructsscenes,controlsbehaviors,andcanrendereachcameraviewindifferentrepresentations,suchascoarse3D,andboundingboxes.Thebenchmarkcoversfirst-andthird-personperspectives,withsynchronizedmulti-viewobservationsavailableforasubsetofepisodes.Groundedintheserecords,PROWBenchevaluatesentitycontrol,long-horizonmemory,and,withtwoVLM-basedmetrics,Logic-RenderAlignmentandInteractionSuccessRate,adherencetotheprescribedtimelineandthevisualrealizationoftimestampedengine-recordedevents.

View arXiv pageView PDFProject pageGitHub0Add to collection

Get this paper in your agent:

hf papers read 2610\.02205

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.02205 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.02205 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.02205 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

VTR-Bench: A Systematic Benchmark for Evaluating Visual Text Rendering in Video Generation

Hugging Face Daily Papers

VTR-Bench introduces a systematic benchmark for evaluating visual text rendering in video generation models, covering 300 prompts across five scenario categories, an automated evaluation pipeline with human alignment, and a Keyframe-Guided Agentic Framework. Experiments on 11 state-of-the-art models reveal widespread difficulties in rendering scene text, with the best model achieving a word error rate of 0.250.