Game2World Engine: Unlocking In-the-Wild Gameplay Videos for World Model Training
Summary
The paper introduces Game2World Engine, a framework for removing UI from gameplay videos to create cleaner training data for video world models, with GameCleaner model achieving state-of-the-art results in UI removal.
View Cached Full Text
Cached at: 08/26/26, 07:13 AM
Paper page - Game2World Engine: Unlocking In-the-Wild Gameplay Videos for World Model Training
Source: https://huggingface.co/papers/2608.24680
Abstract
A framework for removing gameplay UI from videos enables cleaner training data for video world models, improving reward metrics and outperforms mask-based removal methods.
Video games provide a scalable source of training data forvideo world models, offering diverse environments, complex interactions, and abundant in-the-wild gameplay videos. However, raw gameplay footage entangles the game world with screen-space interfaces, introducing game-specific biases and irrelevant dynamics that hinder world-model training. To address this problem, we introduceGameUI-TaxonomyandG2WEngine, a full-stack framework that formalizes gameplay UI grounding and removal.G2WEngineautomatically extracts reusable UI assets from real gameplay videos and synthesizes temporally coherent UI overlays on clean footage. Using this engine, we constructGame2World, comprising 96K synthetic paired videos with precise reconstruction targets and 1,079 in-the-wild clips from 303 games for realistic evaluation. Its asset library contains 5,132 verified UI elements across 21 taxonomy categories, collected from 1,010 representative gameplay frames. Based onGame2World, we proposeGameCleaner, a mask-free gameplay UI removal model that combinesmultimodal semantic understandingwithvideo editingcapabilities. Unlike mask-based methods,GameCleanerdirectly identifies and removes diverse HUD elements while preserving the underlying scene content andtemporal dynamics. In a controlled pilot, world models trained on UI-free gameplay improve overallVideoRewardby 6.83% over those trained on UI-overlaid data. On UI-removal evaluation,GameCleanerachieves an average AAR of 95.36 on synthetic videos, outperforming the strongest temporal mask baseline by 57.3%, and obtains the best in-the-wild AAR of 80.05 with 99.8 background preservation. These results demonstrate the scalable potential of transforming Internet gameplay videos into high-quality world-model training data. Code, dataset, and model will be available at https://github.com/Dongping-Chen/Game2World.
View arXiv pageView PDFGitHub0Add to collection
Get this paper in your agent:
hf papers read 2608\.24680
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2608.24680 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2608.24680 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2608.24680 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
GameWAM: A World Action Model for Video Games
GameWAM introduces the first world-action model for native closed-loop gameplay and GUI control in video games, jointly generating visual observations and executable actions with competitive task success and revealing a source-sensitivity failure mode.
MiniWorld: Democratizing the Training of Video World Models from Scratch
MiniWorld is a reproducible framework for training video world models from scratch using a block-causal Video Diffusion Transformer with Flow Matching, enabling efficient streaming generation and trainable in days on a single 8-GPU server.
Wonder: Video World Model Done Better
Wonder is a general-purpose video world model that enables real-time, camera-controllable world exploration from an image or conditional video. It introduces camera conditioning via dense coordinate fields, a sparse attention memory mechanism, and techniques to improve distillation, allowing minute-scale video generation at 16 FPS.
From Pixels to States: Rethinking Interactive World Models as Game Engines
This paper rethinks interactive world models as game engines by examining four key dimensions—action control, state dynamics, state-observation persistence, and real-time generation—and introduces a scalable data engine for Black Myth: Wukong with over 90 hours of gameplay data to advance state-aware game world modeling.
Agentic Game Development as a Verifiable Trajectory Data Engine for Scaling World Models
This paper proposes using game engines as a verifiable trajectory data engine for scaling world models, introducing RLHEV to combine dense engine signals with human feedback for reinforcement learning post-training.