video-diffusion

Steady-Forcing proposes a memory and training framework to balance spatial stability and motion continuity in long-horizon nature video generation, improving background consistency while sustaining fluid dynamics over multi-minute rollouts.

ByteDance open-sourced Bernini-R, a video diffusion renderer that combines an MLLM-based semantic planner with a DiT-based renderer for unified video generation and editing, achieving top-tier performance on video editing.

τ_0-WM is a unified video-action world model for robotic manipulation that integrates policy learning, video prediction, and action evaluation using a shared video diffusion backbone. It shows superior performance on challenging long-horizon and fine-grained tasks.

LVSA introduces a training-free sparse attention mechanism for video diffusion models, reducing compute up to 3.17x while enabling generation beyond training horizons without quality loss.

VideoMLA replaces per-head KV caches in video diffusion models with a shared low-rank latent and decoupled 3D-RoPE positional keys, reducing per-token KV memory by 92.7% and improving throughput by 1.23x on a B200 while maintaining quality on VBench benchmarks.

This paper identifies a bias in attention weights caused by quantizing keys in KV-cache compression for chunk-wise autoregressive video diffusion, and proposes a per-attention-score correction that removes the bias with negligible overhead, recovering near-BF16 video quality at INT2 quantization.

SCOPE introduces a method for precise action response in FPS games by conditioning transformer blocks in video diffusion models to separate in-scope effects from out-of-scope visual effects without segmentation labels, and presents CrossFPS, a multi-game dataset enabling zero-shot cross-game transfer.

WorldKV is a training-free framework that retrieves and compresses key-value cache chunks to maintain long-term consistency in video diffusion world generation, achieving higher throughput while matching full-memory fidelity.

A novel inference-time method for long video generation using overlapping sliding windows with Tweedie matching and stochastic early-phase sampling to improve temporal consistency and visual quality without additional training.

Q-ARVD is a novel quantization framework to reduce inference costs of autoregressive video diffusion models by addressing frame-wise sensitivity imbalance and weight outlier patterns.

Flash-GRPO improves training efficiency for video diffusion models by addressing temporal variance and gradient inconsistency through iso-temporal grouping and temporal gradient rectification, achieving state-of-the-art alignment quality with substantial training acceleration.

NVIDIA released AnyFlow, the first any-step video diffusion model for text-to-video generation, allowing smooth quality scaling across inference budgets (4 to 50 steps).

VidSplat is a training-free generative reconstruction framework that uses video diffusion priors to recover complete 3D scenes from sparse inputs by synthesizing novel views.

This paper introduces Forcing-KV, a hybrid KV cache compression strategy for autoregressive video diffusion models that separates attention heads into static and dynamic categories, achieving up to 2.82x speedup at 1080P resolution while maintaining output quality.

AnyRecon proposes a scalable framework for 3D reconstruction from arbitrary sparse inputs using a video diffusion model with persistent scene memory and geometry-aware conditioning.

MoZoo is a generative diffusion model that synthesizes high-fidelity animal videos from coarse meshes, using novel attention mechanisms and a synthetic-to-real data pipeline.

This paper introduces Trainable Sparse Attention (VSA), a hardware-efficient sparse attention mechanism that reduces computational costs in video diffusion transformers without compromising performance, enabling more efficient scaling and faster generation.

FramePack is a neural network structure that compresses input frames to fix transformer context length regardless of video length, enabling video diffusion models to process many frames with computation similar to image diffusion and improving batch sizes. It also introduces an anti-drifting sampling method to reduce exposure bias.

This paper presents a scalable open-source pipeline using NVIDIA NeMo for training and inference of Video Foundation Models, addressing challenges in generating high-quality videos with accelerated dataset curation and parallelized training.