RealMe: Immersive Volumetric Video from Minimal Captures

Hugging Face Daily Papers Papers
volumetric-video video-diffusion 4d-reconstruction multi-view computer-vision generative-models

Summary

RealMe 是一个基于高分辨率视频扩散模型的统一框架，能从稀疏、低分辨率的多视角采集生成 2K 分辨率且 4D 一致的沉浸式体积视频，通过布局感知注意力和频域潜空间监督，在 DNA-Rendering 基准和自建 RealVV 数据集上超越现有方法。

This paper addresses the problem of creating immersive volumetric videos from sparse and low-resolution capture resources. Conventional dense multi-view reconstruction methods can produce high-quality volumetric videos, but their reliance on costly capture setups makes them impractical in real-world scenarios. Recent approaches use generative models to relax capture requirements, but still suffer from limited resolution and 4D inconsistency. To overcome these constraints, we propose RealMe, a unified framework built around a high-resolution video diffusion model that enhances limited captures into high-resolution (2K), 4D-consistent videos for volumetric video reconstruction. Specifically, RealMe constructs auxiliary data from capture videos and encodes them into latents along with layout-aware positional embeddings. The video diffusion model adapts factorized spatio-temporal attention to the multi-view capture layout, denoises these latents across viewpoints and time, and decodes them into enhanced videos to refine the 4D representation. This adaptation helps the model capture temporal dynamics and multi-view relationships, leading to more 4D-consistent generation. To reduce the data gap between generated videos and real captures, we apply a latent frequency loss that provides frequency-domain supervision in latent space. Extensive experiments on the public DNA-Rendering benchmark and our self-captured RealVV dataset demonstrate that RealMe is able to create immersive volumetric videos and outperforms state-of-the-art methods in visual fidelity and 4D consistency.
Original Article
View Cached Full Text

Cached at: 10/22/26, 03:37 PM

Paper page - RealMe: Immersive Volumetric Video from Minimal Captures

Source: https://huggingface.co/papers/2610.30023

Abstract

RealMe enhances sparse, low-resolution multi-view captures into high-resolution, 4D-consistent videos using a video diffusion model with layout-aware attention and frequency-domain latent supervision for volumetric reconstruction.

This paper addresses the problem of creating immersive volumetric videos from sparse and low-resolution capture resources. Conventional dense multi-view reconstruction methods can produce high-quality volumetric videos, but their reliance on costly capture setups makes them impractical in real-world scenarios. Recent approaches use generative models to relax capture requirements, but still suffer from limited resolution and 4D inconsistency. To overcome these constraints, we propose RealMe, a unified framework built around a high-resolutionvideo diffusion modelthat enhances limited captures into high-resolution (2K), 4D-consistent videos forvolumetric video reconstruction. Specifically, RealMe constructs auxiliary data from capture videos and encodes them into latents along with layout-awarepositional embeddings. Thevideo diffusion modeladaptsfactorized spatio-temporal attentionto the multi-view capture layout, denoises these latents across viewpoints and time, and decodes them into enhanced videos to refine the 4D representation. This adaptation helps the model capture temporal dynamics and multi-view relationships, leading to more 4D-consistent generation. To reduce the data gap between generated videos and real captures, we apply alatent frequency lossthat provides frequency-domain supervision in latent space. Extensive experiments on the public DNA-Rendering benchmark and our self-captured RealVV dataset demonstrate that RealMe is able to create immersive volumetric videos and outperforms state-of-the-art methods in visual fidelity and4D consistency.

View arXiv pageView PDFProject pageGitHub2Add to collection

Get this paper in your agent:

hf papers read 2610\.30023

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.30023 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.30023 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.30023 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

@FinanceYF5: A Chinese open-source model can reconstruct 3D scenes in real-time from any video. Just one camera, no LiDAR needed, handles over 10,000 frames without crashing, and runs at 20fps on a single GPU. Its benchmark scores surpass traditional optimization-based methods, tested on drone aerial photography, dashcams, and indoor tours. And it's fully open-source.

X AI KOLs Timeline

A Chinese open-source model reconstructs 3D scenes in real-time from a single camera video, no LiDAR required, achieves 20fps on a single GPU with performance superior to traditional optimization methods. Fully open-source.