Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation

Hugging Face Daily Papers Papers
diffusion-transformers monocular-depth-estimation computer-vision flow-matching fine-tuning dense-regression

Summary

Marigold V2 repurposes diffusion transformers for monocular depth estimation via single-step inference and a novel fine-tuning protocol, achieving sharper depth maps and significant improvements on benchmarks like KITTI and ETH3D.

Monocular depth estimation is a ubiquitous yet highly ill-posed computer vision task, with downstream applications in scene reconstruction, computational photography, and robotics, among others. Despite the field's maturity, recent models still struggle to generalize to out-of-distribution inputs and to produce sharp and detailed depth maps. In this paper, we revisit Marigold, a set of techniques for repurposing modern image generation and editing models, powered by the diffusion transformer (DiT) architecture, into state-of-the-art monocular depth estimators. Our recipes target single-step inference from pretrained multi-step flow-matching models, with quantization where needed, preserving model capacity while remaining cheap to run. We analyze the artifacts of naive training and identify two effective remedies: aligning the model's internal representations with semantic features extracted from ground-truth, and adopting a 2-stage fine-tuning protocol built around a novel Sinkhorn-based loss. The results are crisper, cleaner depth maps that generalize well out-of-distribution, with 16-26% improvement in AbsRel over the previous best on KITTI and ETH3D. Qualitatively, our model resolves fur, foliage, and hair-thin edges that have eluded prior models. Furthermore, Marigold V2 achieves state-of-the-art results when applied to other dense regression tasks, such as surface normals estimation and intrinsic image decomposition. Project website: https://hf.co/spaces/huawei-bayerlab/marigold-v2-web
Original Article
View Cached Full Text

Cached at: 09/09/26, 12:31 PM

Paper page - Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation

Source: https://huggingface.co/papers/2609.08084

Abstract

Marigold V2 repurposes diffusion transformers for monocular depth estimation via single-step flow-matching inference, semantic alignment, and a Sinkhorn-based two-stage fine-tuning protocol, yielding sharper out-of-distribution depth maps and strong results on related dense regression tasks.

Monocular depth estimationis a ubiquitous yet highly ill-posed computer vision task, with downstream applications in scene reconstruction, computational photography, and robotics, among others. Despite the field’s maturity, recent models still struggle to generalize to out-of-distribution inputs and to produce sharp and detailed depth maps. In this paper, we revisit Marigold, a set of techniques for repurposing modern image generation and editing models, powered by thediffusion transformer(DiT) architecture, into state-of-the-art monocular depth estimators. Our recipes target single-step inference from pretrained multi-stepflow-matchingmodels, withquantizationwhere needed, preserving model capacity while remaining cheap to run. We analyze the artifacts of naive training and identify two effective remedies: aligning the model’s internal representations with semantic features extracted from ground-truth, and adopting a 2-stage fine-tuning protocol built around a novelSinkhorn-based loss. The results are crisper, cleaner depth maps that generalize well out-of-distribution, with 16-26% improvement in AbsRel over the previous best on KITTI and ETH3D. Qualitatively, our model resolves fur, foliage, and hair-thin edges that have eluded prior models. Furthermore, Marigold V2 achieves state-of-the-art results when applied to other dense regression tasks, such assurface normals estimationandintrinsic image decomposition. Project website: https://hf.co/spaces/huawei-bayerlab/marigold-v2-web

View arXiv pageView PDFProject pageGitHub79Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.08084 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.08084 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.08084 in a Space README.md to link it from this page.

Collections including this paper3

Similar Articles

PointDiT: Pixel-Space Diffusion for Monocular Geometry Estimation

Hugging Face Daily Papers

PointDiT presents a minimalist pixel-space diffusion transformer using a plain ViT architecture for monocular geometry estimation, outperforming complex latent-based models while maintaining simplicity and robustness in ambiguous regions.

chenxwh/depth-anything-v2

Replicate Explore

Depth Anything V2 is a monocular depth estimation model that significantly outperforms V1 in fine-grained details and robustness, offering faster inference and higher accuracy than SD-based models. It is available on Replicate under varying licenses.