Wizard: Manipulate Any World in Real Time via Agentic Control

Hugging Face Daily Papers Papers

Summary

Wizard 提出一种游戏世界模型,通过解耦角色动画、场景操控与游戏逻辑,实现对未见过角色的关节级动作与实时交互控制,支持 24 FPS 交互并对新场景、角色和动作具备零样本泛化能力。

Recent advances in video world models have enabled the generation of increasingly realistic and interactive virtual environments, opening new possibilities for player-customizable gameplay. Agentic frameworks further advance this direction by decoupling explicit game logic and controllable world representations from visual generation. However, existing approaches support only simple game settings and predefined motion patterns, generalize poorly to unseen environments and characters, and offer limited real-time interactivity. In this paper, we introduce Wizard, an agentic game world model that enables real-time, precise control over character motions while generalizing zero-shot across novel scenes, characters, and actions. To achieve this, we construct a diverse dataset spanning 750 scenes and 178 characters, incorporating skeleton and geometry information as structured representations. We leverage these representations as intermediate proxies to condition a video generation model, together with an adaptive masking mechanism that improves alignment between the conditioning signals and generated video rollouts. Furthermore, we introduce an agentic framework that explicitly maintains and updates game states, allowing players to freely configure their virtual worlds and exercise fine-grained control over character motions while preserving visual quality and gameplay consistency. By combining agentic control with structured proxy-based generation, our framework unlocks new possibilities for player-driven game creation with real-time, generalizable, and mechanically consistent world simulation.
Original Article
View Cached Full Text

Cached at: 11/02/26, 03:26 AM

Paper page - Wizard: Manipulate Any World in Real Time via Agentic Control

Source: https://huggingface.co/papers/2610.40339 🪄 Wizard: Manipulate Any World in Real Time via Agentic Control!

Existing game world models produce realistic visuals but remain constrained by familiar environments and predefined motions. Control largely focuses on locomotion, with limited support for articulated actions. Can a world model follow an unseen character swinging a sword, spreading its wings, or transforming mid-battle?

Wizard decouples character animation, scene manipulation, and gameplay logic from visual generation. Agents control articulated skeletons and scene geometry through structured visual proxies, while explicit world states support adaptive boss behavior and mechanics-consistent combat.

We construct 50.3 hours of aligned proxy–video pairs spanning 750 scenes and 178 characters. Character-aware adaptive masking preserves reference appearance, while causal adaptation and few-step distillation enable interaction at 24 FPS, with zero-shot generalization to unseen scenes, characters, and motions.

From world exploration and character summoning to transformation and scene manipulation, Wizard gives players greater freedom to create and control their own game worlds.

Code coming soon!

Similar Articles

Infinite Worlds with Versatile Interactions

Hugging Face Daily Papers

This paper introduces LingBot-World 2.0, an advanced world modeling system with unbounded interaction horizons, real-time 720p 60fps video generation, diverse interactive elements (e.g., attacking, spell-casting), and novel multi-agent behavior control via pilot and director agents, along with a shared multi-player interface.

Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Control

Hugging Face Daily Papers

The paper introduces Zing-0.5, a 5B parameter autoregressive world model for generating playable worlds with real-time user interaction through combined keyboard and text controls. It achieves high performance in navigation tasks and demonstrates low-cost real-time inference at 24 FPS.

Generative World Simulation with DAO and JING

Hugging Face Daily Papers

提出 Generative World Simulation 框架,将持久共享世界状态引擎 DAO 与第一人称交互体验模型 JING 耦合,通过连续交互循环生成可随用户动作演化并保留历史的沉浸式视听体验。