4DThinker: Thinking with 4D Imagery for Dynamic Spatial Understanding

Hugging Face Daily Papers Papers
vision-language-models dynamic-spatial-reasoning 4d-imagery fine-tuning reinforcement-learning

Summary

4DThinker is a new framework that enables vision-language models to perform dynamic spatial reasoning using 4D latent mental imagery. The paper introduces scalable data generation and novel fine-tuning methods, including 4D Reinforcement Learning, to improve model performance on complex dynamic tasks.

Dynamic spatial reasoning from monocular video is essential for bridging visual intelligence and the physical world, yet remains challenging for vision-language models (VLMs). Prior approaches either verbalize spatial-temporal reasoning entirely as text, which is inherently verbose and imprecise for complex dynamics, or rely on external geometric modules that increase inference complexity without fostering intrinsic model capability. In this paper, we present 4DThinker, the first framework that enables VLMs to "think with 4D" through dynamic latent mental imagery, i.e., internally simulating how scenes evolve within the continuous hidden space. Specifically, we first introduce a scalable, annotation-free data generation pipeline that synthesizes 4D reasoning data from raw videos. We then propose Dynamic-Imagery Fine-Tuning (DIFT), which jointly supervises textual tokens and 4D latents to ground the model in dynamic visual semantics. Building on this, 4D Reinforcement Learning (4DRL) further tackles complex reasoning tasks via outcome-based rewards, restricting policy gradients to text tokens to ensure stable optimization. Extensive experiments across multiple dynamic spatial reasoning benchmarks demonstrate that 4DThinker consistently outperforms strong baselines and offers a new perspective toward 4D reasoning in VLMs. Our code is available at https://github.com/zhangquanchen/4DThinker.
Original Article
View Cached Full Text

Cached at: 05/11/26, 02:43 AM

Paper page - 4DThinker: Thinking with 4D Imagery for Dynamic Spatial Understanding

Source: https://huggingface.co/papers/2605.05997 Published on May 7

·

Submitted byhttps://huggingface.co/jankin123

jankinon May 11

#3 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

4DThinker enables vision-language models to perform dynamic spatial reasoning through 4D latent mental imagery, using scalable data generation and novel fine-tuning methods that outperform existing approaches.

Dynamic spatial reasoningfrom monocular video is essential for bridging visual intelligence and the physical world, yet remains challenging forvision-language models(VLMs). Prior approaches either verbalize spatial-temporal reasoning entirely as text, which is inherently verbose and imprecise for complex dynamics, or rely on external geometric modules that increase inference complexity without fostering intrinsic model capability. In this paper, we present 4DThinker, the first framework that enables VLMs to “think with 4D” through dynamic latent mental imagery, i.e., internally simulating how scenes evolve within the continuous hidden space. Specifically, we first introduce a scalable, annotation-freedata generation pipelinethat synthesizes4D reasoningdata from raw videos. We then proposeDynamic-Imagery Fine-Tuning(DIFT), which jointly supervisestextual tokensand 4D latents to ground the model in dynamic visual semantics. Building on this,4D Reinforcement Learning(4DRL) further tackles complex reasoning tasks via outcome-based rewards, restrictingpolicy gradientsto text tokens to ensure stable optimization. Extensive experiments across multipledynamic spatial reasoningbenchmarks demonstrate that 4DThinker consistently outperforms strong baselines and offers a new perspective toward4D reasoningin VLMs. Our code is available at https://github.com/zhangquanchen/4DThinker.

View arXiv pageView PDFGitHub8Add to collection

Get this paper in your agent:

hf papers read 2605\.05997

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.05997 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.05997 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.05997 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

D4RT: Teaching AI to see the world in four dimensions

Google DeepMind Blog

DeepMind introduces D4RT, a unified AI model for dynamic 4D scene reconstruction and tracking that is up to 300x more efficient than previous methods. The model uses a query-based Transformer architecture to solve complex spatial and temporal tasks for robotics and AR applications.

Unfold The World: Factorize 4D Properties in Reinforcing Spatial Reasoning

Hugging Face Daily Papers

This paper introduces FactoSR, a factorized reinforcement learning framework that enhances spatial reasoning in Vision-Language Models by decomposing 4D properties into orthogonal sub-objectives, achieving significant performance boosts on multi-view and video benchmarks.

Beyond Pixels: From Video Priors to 4D Worlds

Hugging Face Daily Papers

This paper introduces Latent-to-4D, a method for direct 4D scene generation from video diffusion latents without retraining across generators, achieving better geometry and temporal stability than cascaded approaches.

Disentangling 3D Modeling from Spatial Reasoning

arXiv cs.LG

This paper proposes DiSR, a framework that separates 3D perception from reasoning by using off-the-shelf perception models to reconstruct explicit 3D evidence and fine-tuning an LLM with LoRA for spatial reasoning, achieving competitive performance with improved interpretability and efficiency.