Self-Geometry: GT-Free and Plug-and-Play Test-Time Adaptation for Geometrically Consistent 3D Vision Foundation Models
Summary
Presents Self-Geometry, a plug-and-play test-time adaptation pipeline that enforces explicit multi-view geometric constraints using 2D pixel correspondences to improve geometrically consistent 3D vision foundation models.
View Cached Full Text
Cached at: 08/13/26, 03:33 PM
Paper page - Self-Geometry: GT-Free and Plug-and-Play Test-Time Adaptation for Geometrically Consistent 3D Vision Foundation Models
Source: https://huggingface.co/papers/2608.10708
Abstract
Self-Geometry improves vision foundation model predictions by enforcing explicit multi-view geometric constraints via test-time adaptation with LoRA, disentangled losses, and angular neighbor sampling.
RecentVision Foundation Models(VFMs) predict depth, camera pose, and pointmap in a single forward pass without per-scene optimization, achieving strong generalization. However, enforcing explicitmulti-view geometric consistency, e.g., throughbundle adjustment, is computationally costly and is thus not imposed during VFM pretraining, so such inconsistency can arise. To address this, implicit self-consistency derived from model outputs (e.g.,pointmaps, features), though enforced at test-time in prior work, delivers inherently limited performance gain, especially on scenes where the pretrained VFM is highly inaccurate. In contrast to this implicit signal, we proposeSelf-Geometry, a plug-and-play test-time adaptation pipeline that directly imposes explicit multi-view geometric constraints using 2D pixel correspondences as pseudo ground-truth. Our proposedSelf-Geometryconsists ofGeometric Disentanglement Optimization, which combines Multi-View Consistency andEpipolar Consistency losses withGradient Disentanglementto prevent gradient conflict;Frame Angular-Neighbor, a view sampler based onSO(3) geodesic distancesfor lightly imposing these constraints; andLightweight TTA, which adapts VFMs viaLoRA. Our method achieves consistent improvements in both pose and geometry estimation across six VFMs (VGGT, π^3, DA3-Giant/Large/Base/Small) and four benchmarks (7Scenes, ETH3D, ScanNet++, HiRoom).
View arXiv pageView PDFProject pageGitHub1Add to collection
Get this paper in your agent:
hf papers read 2608\.10708
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2608.10708 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2608.10708 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2608.10708 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Towards Consistent Video Geometry Estimation
ViGeo is a transformer-based foundation model that recovers dense and consistent 3D geometry from videos using dynamic chunking attention and a completion-based data refinement framework, achieving state-of-the-art performance across multiple tasks.
CGGS: Consistency-Augmented Geometric Gaussian Splatting for Ego-centric 3D Scene Generation
CGGS is a text-to-3D framework that improves geometric consistency and quality in ego-centric 3D scene generation using a multi-stage approach with consistency-augmented loss, layout decoration, and geometric refinement via entropy-based depth loss.
GeoState: Learning View-Decoupled Geometry States for 3D World Modeling
GeoState提出了一种视角解耦的几何表示,将场景动态与相机运动分离以实现预测性3D世界建模,在Waymo上超越先前方法,并可零样本泛化到KITTI和nuScenes。
Drive3R: Block-wise Streaming Geometric Foundation Models for Autonomous Driving
Drive3R 提出一种块状流式几何基础模型,通过块状因果注意力和置信度感知重锚定实现可扩展的长时程多视图 3D 重建,在 nuScenes、KITTI 和 Waymo 上达到新的最先进水平。
SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space
This paper proposes SG-WAM, a self-guided framework for learning geometry-aware action-conditioned world models directly in policy-derived representation space. It achieves state-of-the-art success rates on LIBERO and LIBERO-Plus benchmarks, outperforming strong baselines in real-world evaluations.