Scal3R: Learning Efficient Multi-Relative Pose Query for Scalable Online 3D Reconstruction
Summary
Scal3R improves online 3D reconstruction for long videos by using multi-reference relative pose querying with lightweight tokens and pose-graph optimization, reducing drift and achieving state-of-the-art performance.
View Cached Full Text
Cached at: 09/04/26, 11:57 AM
Paper page - Scal3R: Learning Efficient Multi-Relative Pose Query for Scalable Online 3D Reconstruction
Source: https://huggingface.co/papers/2609.04201
Abstract
Scal3R improves long-video online 3D reconstruction by querying multi-reference relative poses with lightweight tokens and pose-graph optimization, reducing drift without retraining the backbone.
Online 3D reconstructionmodels perform poorly on long videos. This happens because regressing poses relative to a fixed first-frame anchor forces extrapolation far beyond the training distribution. Small drifts accumulateand amplify into significant geometric collapse. However, we observe that per-frame depth remains stable throughout this failure. The backbone’s local geometry remains intact; only the global pose head breaks down. Motivated by this decoupling, we introduce Scal3R. This approach reformulates online reconstruction as multi-referencerelative pose querying. We use lightweightlearnable tokens, which make up about ~1% of the parameters, and inject them into a completelyfrozen backboneviaasymmetric attention. This setup queries poses relative to multiple past keyframes. An onlinepose-graph optimizationsystem withloop closuresuppresses long-range drift. Scal3R reaches convergence in 8 hours on a single GPU. It reduces the averageATEby over 60% on KITTI compared to the online baseline. It also achieves state-of-the-art performance across Virtual KITTI, Sintel, TUM-Dynamic, ScanNet, and 7-Scenes. Project page: https://linjohnss.github.io/scal3r/
View arXiv pageView PDFProject pageAdd to collection
Get this paper in your agent:
hf papers read 2609\.04201
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2609.04201 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2609.04201 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2609.04201 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Lite3R: A Model-Agnostic Framework for Efficient Feed-Forward 3D Reconstruction
Lite3R is a model-agnostic framework that improves the efficiency of transformer-based 3D reconstruction using sparse linear attention and FP8-aware quantization. It reduces latency and memory usage by up to 2.4x while maintaining geometric accuracy on backbones like VGGT and DA3-Large.
AnyRecon: Arbitrary-View 3D Reconstruction with Video Diffusion Model
AnyRecon proposes a scalable framework for 3D reconstruction from arbitrary sparse inputs using a video diffusion model with persistent scene memory and geometry-aware conditioning.
GenRecon: Bridging Generative Priors for Multi-View 3D Scene Reconstruction
GenRecon introduces a method for 3D scene reconstruction that integrates generative 3D priors with multi-view image conditioning, achieving high-fidelity, editable mesh reconstructions of indoor environments and outperforming existing methods by 16%.
TrackCraft3R: Repurposing Video Diffusion Transformers for Dense 3D Tracking
TrackCraft3R repurposes video diffusion transformers for dense 3D tracking from monocular video, using dual-latent representation and temporal RoPE alignment to achieve state-of-the-art performance with 1.3x faster speed and 4.6x less peak memory than prior methods.
4D Human-Scene Reconstruction from Low-Overlap Captures
Proposes StudioRecon, a method for 4D human-scene reconstruction from low-overlap camera captures, using separate priors for backgrounds (video diffusion) and humans (SMPL), with a recursive enhancement module. Accepted to SIGGRAPH 2026, outperforming prior methods by +1.5 to +5.0 dB PSNR.