Programmable World Model

Hugging Face Daily Papers Papers
programmable-world-model video-generation 3d-bounding-boxes executable-programs world-state benchmark generative-renderer

Summary

The Programmable World Model introduces a framework that decouples world-state evolution from visual generation, enabling persistent and controllable environments through executable programs and 3D bounding boxes.

Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent world state and enforcing programmable rules over extended interactions. We introduce Programmable World Model, a framework that decouples world-state evolution from visual observation generation. An agent translates natural-language instructions into executable programs that specify entity states and state-transition rules, enabling direct control over individual entities and their interactions. A lightweight engine executes these programs to update and maintain an explicit, persistent global world state, including off-screen entities and non-visual attributes. To connect world state with visual generation, we introduce state-augmented 3D oriented bounding boxes (OBBs) as an intermediate representation. This representation, together with the target camera trajectory, is deterministically compiled into pixel-aligned spatiotemporal conditioning signals for a pretrained video model serving as the generative renderer. This design allows users to create playable games with predefined mechanics, direct control over individual entities, and persistent world state throughout gameplay. We further introduce CombatStateBench, a benchmark for evaluating programmable world models. On CombatStateBench, our method achieves 94% Count Accuracy and 98% State Accuracy, substantially outperforming existing interactive video world models while supporting coherent long-horizon generation. These results demonstrate the effectiveness of separating explicit state evolution from generative rendering for building persistent, programmable worlds.
Original Article
View Cached Full Text

Cached at: 09/10/26, 02:14 AM

Paper page - Programmable World Model

Source: https://huggingface.co/papers/2609.10540

Abstract

A programmable world model separates explicit state evolution from video generation using executable rules and 3D bounding boxes to maintain persistent, controllable environments.

Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent world state and enforcing programmable rules over extended interactions. We introduceProgrammable World Model, a framework that decouples world-state evolution from visual observation generation. An agent translates natural-language instructions intoexecutable programsthat specifyentity statesandstate-transition rules, enabling direct control over individual entities and their interactions. A lightweight engine executes these programs to update and maintain an explicit, persistentglobal world state, including off-screen entities and non-visual attributes. To connect world state with visual generation, we introduce state-augmented3D oriented bounding boxes(OBBs) as an intermediate representation. This representation, together with the target camera trajectory, is deterministically compiled into pixel-alignedspatiotemporal conditioning signalsfor apretrained video modelserving as thegenerative renderer. This design allows users to create playable games with predefined mechanics, direct control over individual entities, and persistent world state throughout gameplay. We further introduceCombatStateBench, a benchmark for evaluatingprogrammable world models. OnCombatStateBench, our method achieves 94% Count Accuracy and 98% State Accuracy, substantially outperforming existing interactive video world models while supporting coherent long-horizon generation. These results demonstrate the effectiveness of separating explicit state evolution from generative rendering for building persistent, programmable worlds.

View arXiv pageView PDFProject pageGitHubAdd to collection

Get this paper in your agent:

hf papers read 2609\.10540

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.10540 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.10540 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.10540 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Code World Model: Coding Agent as World Brain

Hugging Face Daily Papers

The paper introduces Code World Model, a framework that combines language models for reasoning and coding with video models for visual rendering to simulate persistent world evolution in an open-ended manner.

PatchWorld: Gradient-Free Optimization of Executable World Models

arXiv cs.CL

PatchWorld introduces a gradient-free framework that transforms offline trajectories into executable Python world models via counterexample-guided code repair, enabling interpretable and inspectable belief-state programs for planning in partially observable environments.