Puffin-World: Scaling a Unified Multimodal Model with Native 3D World States

Hugging Face Daily Papers Papers
multimodal-model 3d-world-generation physics-simulation geometric-reconstruction generative-process dataset-scaling

Summary

Puffin-World is a unified multimodal framework that integrates physical understanding, spatial simulation, and 3D world generation using an Omni-Camera representation, scaled with the Puffin-16M dataset.

We propose Puffin-World, a unified multimodal architecture that integrates physical understanding, spatial simulation, and 3D world generation and reconstruction without relying on external offline modules. To reliably construct and interact with 3D worlds, our framework jointly models three native world states: physics (gravity field and latitude), geometry (depth), and appearance (image), together with a unified Omni-Camera representation that supports diverse tasks and flexible motions. Beyond modeling these states, we introduce a strategy for propagating physical dynamics across future frames. By grounding absolute camera properties in the real world, Puffin-World enables physically consistent and visually stable world generation. We further couple appearance and geometry within a single generative process, jointly synthesizing each future view and reconstructing its underlying geometry. This unified paradigm enables interleaved closed-loop applications requiring synergy across multiple tasks, including mimic and self-calibrated world exploration. To scale Puffin-World to complex scenarios, we construct Puffin-16M, comprising 15 million vision-language-camera triplets and 1 million trajectories featuring various and challenging motions. To foster further research in this area, we released the code, models, and datasets.
Original Article
View Cached Full Text

Cached at: 09/04/26, 07:57 AM

Paper page - Puffin-World: Scaling a Unified Multimodal Model with Native 3D World States

Source: https://huggingface.co/papers/2609.04196

Abstract

Puffin-World is a unified multimodal framework that jointly models physics, geometry, and appearance for physically consistent 3D world generation, reconstruction, and closed-loop exploration.

We propose Puffin-World, a unifiedmultimodal architecturethat integratesphysical understanding,spatial simulation, and3D world generationand reconstruction without relying on external offline modules. To reliably construct and interact with 3D worlds, our framework jointly models three native world states: physics (gravity fieldand latitude), geometry (depth), and appearance (image), together with a unifiedOmni-Camera representationthat supports diverse tasks and flexible motions. Beyond modeling these states, we introduce a strategy for propagating physical dynamics across future frames. By grounding absolute camera properties in the real world, Puffin-World enables physically consistent and visually stable world generation. We further couple appearance and geometry within a singlegenerative process, jointly synthesizing each future view and reconstructing its underlying geometry. This unified paradigm enables interleaved closed-loop applications requiring synergy across multiple tasks, including mimic and self-calibrated world exploration. To scale Puffin-World to complex scenarios, we construct Puffin-16M, comprising 15 millionvision-language-camera tripletsand 1 million trajectories featuring various and challenging motions. To foster further research in this area, we released the code, models, and datasets.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2609\.04196

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### ACERobotics/Puffin-World Image-to-3D• Updatedabout 3 hours ago • 8

Datasets citing this paper1

#### KangLiao/Puffin-16M Preview• Updatedabout 3 hours ago • 887 • 8

Spaces citing this paper1

Collections including this paper1

Similar Articles

Cosmos 3: Omnimodal World Models for Physical AI

Hugging Face Daily Papers

Cosmos 3 is a family of omnimodal world models from NVIDIA that jointly processes language, image, video, audio, and action sequences using a unified mixture-of-transformers architecture, achieving state-of-the-art performance in understanding and generation tasks for Physical AI.

PixWorld: Unifying 3D Scene Generation and Reconstruction in Pixel Space

Hugging Face Daily Papers

PixWorld presents a unified pixel-space diffusion approach for 3D scene reconstruction and generation, overcoming limitations of latent-space methods by using direct image-level supervision and geometry-aware feature alignment. The method outperforms prior generation methods and matches state-of-the-art reconstruction methods.

PanoWorld: Real-World Panoramic Generation

Hugging Face Daily Papers

PanoWorld proposes a method for long-range memory in panoramic world models using rotation-equivariant representations, with a three-stage training pipeline and a new large-scale dataset World360. The model outperforms alternatives by a large margin.