Walking in the Implicit: Interactive World Exploration via Neural Scene Representation

Hugging Face Daily Papers Papers
neural-scene-representation interactive-video-generation diffusion-transformer implicit-state transformer-vae camera-trajectory long-horizon-consistency

Summary

NeuWorld is a new interactive video generation system that uses compact neural implicit scene representations and a transformer VAE with diffusion transformer for trajectory-conditioned rendering, achieving long-horizon consistency.

Interactive video generation systems for camera-controlled world exploration roll out growing sequences of latent video frames, entangling state transition with high-frequency observation synthesis. We propose Walking in the Implicit, a scene-centric paradigm that changes the rollout variable from frame latents to a fixed-length, renderable implicit state, termed Neural Implicit Scene (NIS). This factorizes interactive generation into stochastic transition of a compact scene state and deterministic pose-conditioned rendering given the sampled state. We instantiate this paradigm as NeuWorld: a transformer VAE learns locally anchored NIS from sparse posed frames, and a diffusion transformer evolves NIS conditioned on future camera trajectories and geometry-aware retrieved history. By reusing the VAE encoder as a unified conditioner, NeuWorld maps camera, reference-image, and history cues into the same NIS modality, avoiding external heterogeneous encoders. Trained from scratch on public posed-view data without pretrained video backbones or auxiliary 3D reconstructors, NeuWorld achieves strong long-horizon consistency with favorable inference efficiency.
Original Article
View Cached Full Text

Cached at: 06/30/26, 07:35 AM

Paper page - Walking in the Implicit: Interactive World Exploration via Neural Scene Representation

Source: https://huggingface.co/papers/2606.30045

Abstract

NeuWorld enables efficient interactive video generation by representing scenes as compact neural implicit states and using a transformer VAE with diffusion transformer for trajectory-conditioned rendering.

Interactive video generation systems for camera-controlled world exploration roll out growing sequences oflatent video frames, entangling state transition with high-frequency observation synthesis. We propose Walking in the Implicit, a scene-centric paradigm that changes the rollout variable from frame latents to a fixed-length, renderableimplicit state, termedNeural Implicit Scene(NIS). This factorizes interactive generation into stochastic transition of a compact scene state and deterministicpose-conditioned renderinggiven the sampled state. We instantiate this paradigm as NeuWorld: atransformer VAElearns locally anchored NIS from sparse posed frames, and adiffusion transformerevolves NIS conditioned on futurecamera trajectoriesand geometry-aware retrieved history. By reusing theVAE encoderas aunified conditioner, NeuWorld maps camera, reference-image, and history cues into the same NIS modality, avoiding external heterogeneous encoders. Trained from scratch on public posed-view data without pretrained video backbones or auxiliary 3D reconstructors, NeuWorld achieves stronglong-horizon consistencywith favorable inference efficiency.

View arXiv pageView PDFProject pageGitHub17Add to collection

Get this paper in your agent:

hf papers read 2606\.30045

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.30045 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.30045 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.30045 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Latent Spatial Memory for Video World Models

Hugging Face Daily Papers

This paper introduces latent spatial memory for video world models, storing 3D scene information directly in diffusion latent space to avoid costly pixel-space reconstruction. The proposed Mirage framework achieves up to 10.57x faster generation and 55x memory reduction while achieving state-of-the-art performance on WorldScore and RealEstate10K.

Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation

Hugging Face Daily Papers

Qwen-RobotWorld is a language-conditioned video world model that predicts future visual trajectories across multiple robotic domains using a double-stream diffusion transformer and an 8.6M video-text corpus. It unifies embodied world modeling for robotic manipulation, autonomous driving, indoor navigation, and human-to-robot transfer, achieving top benchmarks on EWMBench and DreamGen Bench.

The DAWN of World-Action Interactive Models

Hugging Face Daily Papers

This paper introduces DAWN, a latent generative baseline for World-Action Interactive Models (WAIMs) that jointly models scene evolution and action generation through recursive refinement, achieving strong long-horizon planning in autonomous driving scenarios.

Learning Visual Feature-Based World Models via Residual Latent Action

Hugging Face Daily Papers

This paper introduces RLA-WM, a visual feature-based world model that leverages residual latent actions and flow matching to efficiently predict future visual states. The method outperforms existing video-diffusion and feature-based approaches while enabling novel robot learning techniques from offline, actionless demonstration videos.