From Pixels to States: Rethinking Interactive World Models as Game Engines

Hugging Face Daily Papers Papers
interactive-world-models game-engines video-generation game-states data-engine black-myth-wukong ai-research

Summary

This paper rethinks interactive world models as game engines by examining four key dimensions—action control, state dynamics, state-observation persistence, and real-time generation—and introduces a scalable data engine for Black Myth: Wukong with over 90 hours of gameplay data to advance state-aware game world modeling.

Building interactive worlds that respond coherently to player actions has long been a shared goal of computer graphics, games, and artificial intelligence. Recent video generative models provide a data-driven route toward this goal by predicting future observations conditioned on user actions, and are increasingly regarded as potential next-generation game engines. Realizing a genuinely interactive game world, however, requires interaction outcomes that follow rules over evolving game conditions, consequences that persist over long horizons, and a generation loop that operates in real time. Conventional game engines realize these properties through a recurrent action-state-observation loop, in which player actions update an explicit game state according to predefined rules and observations are rendered from the resulting state. Taking this loop as an organizing lens, this paper examines interactive game world modeling along four dimensions: player action control, game state dynamics, state-observation persistence, and real-time interactive generation. For each dimension, we start from the capabilities required by an interactive game world, group existing approaches into representative families, and discuss the strengths and trade-offs of each family. Complementing this analysis, we present a scalable data engine for Black Myth: Wukong that collects over 90 hours of gameplay with frame-aligned player actions, ground-truth game states, and visual observations, together with structured and semantic annotations, as a resource for state-aware game world modeling. We hope this paper offers a clear picture of where the field stands and fosters progress toward interactive game worlds.
Original Article
View Cached Full Text

Cached at: 07/20/26, 09:42 AM

Paper page - From Pixels to States: Rethinking Interactive World Models as Game Engines

Source: https://huggingface.co/papers/2607.14076

Abstract

Buildinginteractiveworldsthatrespondcoherentlytoplayeractionshaslongbeenasharedgoalofcomputergraphics,games,andartificialintelligence.Recentvideogenerativemodelsprovideadata-drivenroutetowardthisgoalbypredictingfutureobservationsconditionedonuseractions,andareincreasinglyregardedaspotentialnext-generationgameengines.Realizingagenuinelyinteractivegameworld,however,requiresinteractionoutcomesthatfollowrulesoverevolvinggameconditions,consequencesthatpersistoverlonghorizons,andagenerationloopthatoperatesinrealtime.Conventionalgameenginesrealizethesepropertiesthrougharecurrentaction-state-observationloop,inwhichplayeractionsupdateanexplicitgamestateaccordingtopredefinedrulesandobservationsarerenderedfromtheresultingstate.Takingthisloopasanorganizinglens,thispaperexaminesinteractivegameworldmodelingalongfourdimensions:playeractioncontrol,gamestatedynamics,state-observationpersistence,andreal-timeinteractivegeneration.Foreachdimension,westartfromthecapabilitiesrequiredbyaninteractivegameworld,groupexistingapproachesintorepresentativefamilies,anddiscussthestrengthsandtrade-offsofeachfamily.Complementingthisanalysis,wepresentascalabledataengineforBlackMyth:Wukongthatcollectsover90hoursofgameplaywithframe-alignedplayeractions,ground-truthgamestates,andvisualobservations,togetherwithstructuredandsemanticannotations,asaresourceforstate-awaregameworldmodeling.Wehopethispaperoffersaclearpictureofwherethefieldstandsandfostersprogresstowardinteractivegameworlds.

View arXiv pageView PDFGitHub409Add to collection

Get this paper in your agent:

hf papers read 2607\.14076

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.14076 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.14076 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.14076 in a Space README.md to link it from this page.

Collections including this paper2

Similar Articles

Infinite Worlds with Versatile Interactions

Hugging Face Daily Papers

This paper introduces LingBot-World 2.0, an advanced world modeling system with unbounded interaction horizons, real-time 720p 60fps video generation, diverse interactive elements (e.g., attacking, spell-casting), and novel multi-agent behavior control via pilot and director agents, along with a shared multi-player interface.

Multiplayer Interactive World Models with Representation Autoencoders

Hugging Face Daily Papers

This paper introduces MIRA, the first large-scale multiplayer world model for highly dynamic physics-based environments, trained on 10,000 hours of Rocket League gameplay. The 5-billion-parameter latent diffusion model generates stable four-player rollouts in real time, with distributional quality holding steady for hours.