Introducing GWM Worlds 2, a Playable World Model | Runway

# GWM Worlds 2: An Interactive World Model with Real-Time Control **TL;DR:** Runway has introduced GWM Worlds 2, an interactive world model that can be controlled in real-time. It uses a "World Prompt" mechanism to separate persistent and dynamic elements of a scene, supports frame-by-frame real-time generation of 720p/24fps video and 48kHz audio, allows users to directly control the environment through text actions, and can be integrated with agents to autonomously perform tasks. ## From Simulation to Real-Time Control Last December, we introduced GWM Worlds—a world model for real-time environment simulation. It maintained spatial coherence through long-sequence motion, which is the difficult part. Today, we present GWM Worlds 2. Video and audio generation transforms into a real-time controllable simulation experience. ## Core Mechanism: World Prompt Rich control requires a mechanism to describe the world. Our approach starts with a segmentation: distinguishing persistent elements from dynamic changes. We call this the "World Prompt." * **Persistent Part (Scene):** The actionable subjects constituting the scene and the rules they follow. The first frame lays the foundation, and thereafter the world continues to run. * **Event Stream:** Timestamped text actions that can target specific subjects or freely affect the entire scene. Camera movement is provided frame-by-frame as a separate input and maintains continuity. ## Technical Implementation and Model Evolution To achieve all this, we fine-tuned a base audio-video model according to the World Prompt format, making it follow the corresponding structure. This model originally had a bidirectional structure and ran too slowly for real-time interaction. Therefore, we transformed it into a duration-free, autoregressive model through post-training. GWM Worlds 2 can generate **720p resolution**, **24 frames per second** video and **48kHz audio**, rendered in real-time frame by frame. All subsequent demonstrations are completed through real-time control in a single continuous session. ## Real-Time Control and Interactive Features 1. **Real-Time Control:** Nothing is a pre-rendered clip. As long as you keep inputting, the world will continue and expand. 2. **Easy Operation:** Common actions can be bound to keys to improve control efficiency. Actions do not have to target a specific character; sending a command to the scene can trigger a flood or a sandstorm. 3. **Dynamic Start:** You can also start a scene from a video (not just a static image), with environment and audio transitioning seamlessly. 4. **Agent Control:** Agents can also take over control—give it a goal, and it will act autonomously. In an initial test, the instruction to sequentially go to a red flag and a blue flag was completed in one fluid motion. 5. **Multi-User Collaboration:** Different users can play different roles within the same session. 6. **Convenient Presets:** GWM Worlds 2 does not require tedious scene configuration. Describe the world in one line of text, and you get an instant, ready-to-play preset environment. **GWM Worlds 2. Build as you play.** Source: Introducing GWM Worlds 2, a Playable World Model | Runway (https://www.youtube.com/watch?v=pmjyB-lc4KQ)