MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffold
Summary
MoVerse generates real-time interactive video from single images by creating 360° panoramas and 3D Gaussian scaffolds, enabling efficient rendering through diffusion-based techniques.
View Cached Full Text
Cached at: 06/12/26, 02:52 AM
Paper page - MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffold
Source: https://huggingface.co/papers/2606.13376
Abstract
MoVerse generates real-time interactive video from single images by creating 360° panoramas and 3D Gaussian scaffolds, enabling efficient rendering through diffusion-based techniques.
We present MoVerse, a real-timevideo world modelthat creates an interactively navigable scene from a single narrow-field-of-view image. This setting is challenging because the input observes only a small fraction of the environment, while interactive roaming requires a complete surrounding world, persistent geometry, controllable camera motion, and temporally coherent high-fidelity observations. MoVerse addresses this problem by separating world construction from observation rendering. It first expands the input into a gravity-aligned 360^circ panorama withtopology-aware diffusion, closing the missing field of view before 3D reasoning. It then lifts the panorama into a persistent3D Gaussian scaffoldusingpanoramic geometry-aware residual prediction, yielding a dense and directly renderable spatial memory. Finally, aGaussian-conditioned video renderertranslates scaffold renderings along user-specified camera trajectories into photorealistic video. To make this renderer practical for interaction, we train abidirectional diffusionteacher for high-quality conditional rendering and distill it into acausal autoregressive studentfor bounded-latency streaming. This design combines the controllability and long-range consistency of explicit 3D representations with the perceptual quality of generative video models. MoVerse supportsreal-time scene roamingat 8~FPS on a single NVIDIA RTX~4090 GPU, demonstrating a practical path toward single-image world creation with interactive video output.
View arXiv pageView PDFProject pageGitHubAdd to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.13376 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.13376 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.13376 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video
AlayaVista is a camera-controllable streaming video world model that decouples panoramic scene evolution from perspective video synthesis, supported by the large-scale MUGEN dataset for high-fidelity interactive world modeling.
HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds
HY-World 2.0 is a multi-modal world model framework that generates high-fidelity 3D Gaussian Splatting scenes from text, images, and videos through specialized modules for panorama generation, trajectory planning, and scene composition, achieving state-of-the-art performance among open-source approaches.
@vincieye: Monocular video to 3D in real time? MAGiSt3R uses multiple agents & a merging model (MAGMA) to reconstruct scenes from …
MAGiSt3R is a multi-agent feed-forward framework that achieves real-time 3D reconstruction from monocular RGB videos at 10 FPS, using a merging model (MAGMA) to combine local point maps and pose graph optimization to reduce drift.
Pantheon360: Taming Digital Twin Generation via 3D-Aware 360° Video Diffusion
Pantheon360 introduces a 3D-aware 360° video diffusion framework that uses an explicit 3D cache to enforce geometric consistency, enabling high-fidelity digital twin generation from sparse 360° inputs.
PanoWorld: Real-World Panoramic Generation
PanoWorld proposes a method for long-range memory in panoramic world models using rotation-equivariant representations, with a three-stage training pipeline and a new large-scale dataset World360. The model outperforms alternatives by a large margin.