Why Do Video Diffusion Models Violate Physics? Unveiling the Flaws in Attention Mechanisms

Hugging Face Daily Papers Papers
video-diffusion-models physics-violations attention-mechanisms interpretability rope-analysis architectural-modification

Summary

This paper presents an interpretability study on video diffusion models, revealing that Rotary Position Embedding (RoPE) induces excessive spatial attention decay, causing physics violations, and proposes a lightweight architectural modification to enhance physical coherence in generated videos.

Despite impressive visual quality, state-of-the-art video diffusion models often generate content that violates real-world physical laws. While existing solutions rely on external priors or specialized data, we investigate the root cause by exploring the internal mechanisms of these models. Specifically, we present the first interpretability study on the ''motion planning'' process of text-to-video diffusion models, revealing how motion trajectories form during early denoising stages. Building upon the ''first shape, then details'' finding, we combine cross-attention trajectory patterns with causal head contributions to identify a specific subset of attention heads driving motion planning. Further, our self-attention analysis shows that Rotary Position Embedding (RoPE) induces excessive spatial attention decay. This causes early candidate regions to prematurely lock into physically implausible positions, suppressing reasonable trajectories in adjacent frames and triggering generation failure modes. To address this fundamental flaw, we propose a lightweight architectural modification that scales the frequency of RoPE across different denoising steps. This strategy reduces excessive attention decay, helping the model explore better candidate regions to establish coherent physical motion. Finally, training-free and training-based experiments confirm the effectiveness of our approach in enhancing the physical commonsense of generated videos.
Original Article
View Cached Full Text

Cached at: 09/22/26, 03:25 AM

Paper page - Why Do Video Diffusion Models Violate Physics? Unveiling the Flaws in Attention Mechanisms

Source: https://huggingface.co/papers/2609.23658

Abstract

Despiteimpressivevisualquality,state-of-the-artvideodiffusionmodelsoftengeneratecontentthatviolatesreal-worldphysicallaws.Whileexistingsolutionsrelyonexternalpriorsorspecializeddata,weinvestigatetherootcausebyexploringtheinternalmechanismsofthesemodels.Specifically,wepresentthefirstinterpretabilitystudyonthe’‘motionplanning’‘processoftext-to-videodiffusionmodels,revealinghowmotiontrajectoriesformduringearlydenoisingstages.Buildinguponthe’‘firstshape,thendetails’’finding,wecombinecross-attentiontrajectorypatternswithcausalheadcontributionstoidentifyaspecificsubsetofattentionheadsdrivingmotionplanning.Further,ourself-attentionanalysisshowsthatRotaryPositionEmbedding(RoPE)inducesexcessivespatialattentiondecay.Thiscausesearlycandidateregionstoprematurelylockintophysicallyimplausiblepositions,suppressingreasonabletrajectoriesinadjacentframesandtriggeringgenerationfailuremodes.Toaddressthisfundamentalflaw,weproposealightweightarchitecturalmodificationthatscalesthefrequencyofRoPEacrossdifferentdenoisingsteps.Thisstrategyreducesexcessiveattentiondecay,helpingthemodelexplorebettercandidateregionstoestablishcoherentphysicalmotion.Finally,training-freeandtraining-basedexperimentsconfirmtheeffectivenessofourapproachinenhancingthephysicalcommonsenseofgeneratedvideos.

View arXiv pageView PDFProject pageGitHub1Add to collection

Get this paper in your agent:

hf papers read 2609\.23658

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.23658 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.23658 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.23658 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

The Attention Triangle in Audio-Video Models

arXiv cs.AI

This paper investigates semantic leakage in audio-video diffusion models through the 'attention triangle' of cross-attention mechanisms, and presents methods to enhance semantic grounding during generation.

YoCausal: How Far is Video Generation from World Model? A Causality Perspective

Hugging Face Daily Papers

This paper introduces YoCausal, a benchmark based on the Violation of Expectation paradigm from cognitive science, to evaluate whether video diffusion models truly understand causality or merely overfit to temporal patterns. Evaluation of 13 state-of-the-art models reveals a significant gap compared to human-level causal cognition.