interactive-control

WorldSonus is an interactive video-to-audio framework that brings real-time spatial sound synthesis to world models, using a streaming causal autoregressive diffusion architecture with a real-time factor of 0.41, chunk-indexed prompt scheduling for mid-stream sound control, and stereo supervision for spatial alignment.

Runway has released GWM Worlds 2, an interactive world model with real-time control capabilities, enabling the generation of 720p/24fps video and 48kHz audio on the fly. It allows users to manipulate the environment via text commands and integrate agents for task execution.