MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generation

Hugging Face Daily Papers Papers
multi-view point-tracking 4d-video diffusion-models geometric-consistency video-generation

Summary

MVTrack4Gen introduces a training framework that uses multi-view point tracking as geometric supervision to enhance motion-aware diffusion models, achieving state-of-the-art geometric consistency and motion fidelity in novel-view video generation from monocular video.

Synthesizing a novel-view video from a monocular reference video along a target camera trajectory requires both geometric consistency and motion fidelity with respect to the reference video. Existing methods based on explicit 3D representations are limited by the accuracy of off-the-shelf reconstruction modules, which often produce inaccurate geometry for dynamic objects in monocular videos. In contrast, camera-conditioning-only methods can achieve high visual quality but often struggle to preserve geometric and motion consistency. In this work, we introduce MVTrack4Gen (Multi-View point Tracking for Novel-View Generation), a motion-aware training framework that leverages multi-view point tracking as an additional geometric and motion supervision signal for camera-conditioning-only novel-view video diffusion models. Our key finding is that specific attention layers encode strong correspondence cues, where query features attend to key features at geometrically corresponding locations across views and over time, and the misalignment of these correspondences causes motion inconsistency. Based on this observation, we route these features into an auxiliary multi-view tracking head and jointly train the diffusion model with a point-tracking objective. By explicitly strengthening these motion-aware correspondences, MVTrack4Gen improves existing models to better follow the motion in the reference view and maintain cross-view geometric consistency. Across diverse benchmarks, our method achieves state-of-the-art geometric consistency and competitive camera accuracy.
Original Article
View Cached Full Text

Cached at: 06/25/26, 09:10 AM

Paper page - MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generation

Source: https://huggingface.co/papers/2606.26087 Authors:

,

,

,

,

,

,

,

,

,

Abstract

A novel-view video synthesis method that enhances motion-aware diffusion models through multi-view point tracking supervision to improve geometric consistency and motion fidelity.

Synthesizing a novel-view video from a monocular reference video along a target camera trajectory requires bothgeometric consistencyandmotion fidelitywith respect to the reference video. Existing methods based on explicit 3D representations are limited by the accuracy of off-the-shelf reconstruction modules, which often produce inaccurate geometry for dynamic objects in monocular videos. In contrast,camera-conditioning-onlymethods can achieve high visual quality but often struggle to preserve geometric and motion consistency. In this work, we introduce MVTrack4Gen (Multi-View point Trackingfor Novel-View Generation), a motion-aware training framework that leveragesmulti-view point trackingas an additional geometric and motion supervision signal forcamera-conditioning-onlynovel-view videodiffusion models. Our key finding is that specificattention layersencode strongcorrespondence cues, where query features attend to key features at geometrically corresponding locations across views and over time, and the misalignment of these correspondences causes motion inconsistency. Based on this observation, we route these features into anauxiliary multi-view tracking headand jointly train the diffusion model with a point-tracking objective. By explicitly strengthening these motion-aware correspondences, MVTrack4Gen improves existing models to better follow the motion in the reference view and maintaincross-view geometric consistency. Across diverse benchmarks, our method achieves state-of-the-artgeometric consistencyand competitive camera accuracy.

View arXiv pageView PDFProject pageGitHub20Add to collection

Get this paper in your agent:

hf papers read 2606\.26087

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.26087 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.26087 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.26087 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles