Walking in the Implicit: Interactive World Exploration via Neural Scene Representation

Hugging Face Daily Papers Papers
neural-scene-representation interactive-video-generation diffusion-transformer implicit-state transformer-vae camera-trajectory long-horizon-consistency

Summary

NeuWorld is a new interactive video generation system that uses compact neural implicit scene representations and a transformer VAE with diffusion transformer for trajectory-conditioned rendering, achieving long-horizon consistency.

Interactive video generation systems for camera-controlled world exploration roll out growing sequences of latent video frames, entangling state transition with high-frequency observation synthesis. We propose Walking in the Implicit, a scene-centric paradigm that changes the rollout variable from frame latents to a fixed-length, renderable implicit state, termed Neural Implicit Scene (NIS). This factorizes interactive generation into stochastic transition of a compact scene state and deterministic pose-conditioned rendering given the sampled state. We instantiate this paradigm as NeuWorld: a transformer VAE learns locally anchored NIS from sparse posed frames, and a diffusion transformer evolves NIS conditioned on future camera trajectories and geometry-aware retrieved history. By reusing the VAE encoder as a unified conditioner, NeuWorld maps camera, reference-image, and history cues into the same NIS modality, avoiding external heterogeneous encoders. Trained from scratch on public posed-view data without pretrained video backbones or auxiliary 3D reconstructors, NeuWorld achieves strong long-horizon consistency with favorable inference efficiency.
Original Article
View Cached Full Text

Cached at: 06/30/26, 07:35 AM

Paper page - Walking in the Implicit: Interactive World Exploration via Neural Scene Representation

Source: https://huggingface.co/papers/2606.30045

Abstract

NeuWorld enables efficient interactive video generation by representing scenes as compact neural implicit states and using a transformer VAE with diffusion transformer for trajectory-conditioned rendering.

Interactive video generation systems for camera-controlled world exploration roll out growing sequences oflatent video frames, entangling state transition with high-frequency observation synthesis. We propose Walking in the Implicit, a scene-centric paradigm that changes the rollout variable from frame latents to a fixed-length, renderableimplicit state, termedNeural Implicit Scene(NIS). This factorizes interactive generation into stochastic transition of a compact scene state and deterministicpose-conditioned renderinggiven the sampled state. We instantiate this paradigm as NeuWorld: atransformer VAElearns locally anchored NIS from sparse posed frames, and adiffusion transformerevolves NIS conditioned on futurecamera trajectoriesand geometry-aware retrieved history. By reusing theVAE encoderas aunified conditioner, NeuWorld maps camera, reference-image, and history cues into the same NIS modality, avoiding external heterogeneous encoders. Trained from scratch on public posed-view data without pretrained video backbones or auxiliary 3D reconstructors, NeuWorld achieves stronglong-horizon consistencywith favorable inference efficiency.

View arXiv pageView PDFProject pageGitHub17Add to collection

Get this paper in your agent:

hf papers read 2606\.30045

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.30045 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.30045 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.30045 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Latent Spatial Memory for Video World Models

Hugging Face Daily Papers

This paper introduces latent spatial memory for video world models, storing 3D scene information directly in diffusion latent space to avoid costly pixel-space reconstruction. The proposed Mirage framework achieves up to 10.57x faster generation and 55x memory reduction while achieving state-of-the-art performance on WorldScore and RealEstate10K.

World-in-Context: Interactive Video World Modeling with In-Context Conditioning

Papers with Code Trending

World-in-Context introduces an interactive video world modeling framework that unifies view, action, and camera controls as explicit tokens within a shared self-attention space via In-Context Conditioning, along with Interaction-Transport RoPE for temporal continuity. Its 5B model matches the navigation performance of the 22B EchoWM on WBench while achieving a 93.8% perspective-switching success rate versus 18.8% for existing 5B baselines.

World in World: Explore the World with World Models

Hugging Face Daily Papers

The paper presents World in World, a training-free interface that enables flexible camera and time control in frozen autoregressive video world models by using correspondence-guided queries and evidence-wise attention guidance.

Image2Sim: Scaling Embodied Navigation via Generative Neural Simulator

Hugging Face Daily Papers

Image2Sim is a neural simulation framework that creates high-fidelity interactive environments from RGB-D images, enabling scalable training for embodied navigation agents. It generates nearly 20K scenes and over 10 million training samples, showing strong benchmark improvements and effective real-world zero-shot transfer.