Streaming Video Editing with Easy Adaptation

Hugging Face Daily Papers Papers
video-editing diffusion-models streaming auto-regressive controllable-generation zero-shot

Summary

This paper introduces SVEET, a framework for high-quality streaming video editing that leverages a pretrained video diffusion model to enable auto-regressive editing with real-time performance on a single GPU.

In this paper, we propose SVEET, a framework that requires merely training on a pretrained bidirectional video diffusion model but supports high-quality streaming video editing in an auto-regressive fashion. To tackle this problem, we first systematically revisit existing video-to-video diffusion approaches and identify two key principles for such streaming adaptation: backbone feature disentanglement and conditional frame independence. Building on these insights, we develop a novel paradigm for controllable video generation. At its core, an auxiliary model branch encodes source video inputs with temporally independent self-attention, and the intermediate features are injected into the corresponding backbone blocks for streaming-compatible control. Moreover, to bridge the discrepancy between the feature spaces of bidirectional and streaming models, we propose a decoupled training scheme that explicitly enforces the orthogonality between the optimization directions of video controllability and model causality. Such disentanglement ensures compatibility between the two objectives at inference and facilitates smooth zero-shot knowledge transfer across heterogeneous backbone architectures. Extensive experiments demonstrate that SVEET achieves superior editing quality while maintaining real-time performance, attaining 15 FPS on a single H100 GPU 17 without any auxiliary acceleration techniques. Codes are available at https://github.com/YujiaHu1109/SVEET.
Original Article
View Cached Full Text

Cached at: 09/22/26, 11:26 AM

Paper page - Streaming Video Editing with Easy Adaptation

Source: https://huggingface.co/papers/2609.24788

Abstract

Inthispaper,weproposeSVEET,aframeworkthatrequiresmerelytrainingonapretrainedbidirectionalvideodiffusionmodelbutsupportshigh-qualitystreamingvideoeditinginanauto-regressivefashion.Totacklethisproblem,wefirstsystematicallyrevisitexistingvideo-to-videodiffusionapproachesandidentifytwokeyprinciplesforsuchstreamingadaptation:backbonefeaturedisentanglementandconditionalframeindependence.Buildingontheseinsights,wedevelopanovelparadigmforcontrollablevideogeneration.Atitscore,anauxiliarymodelbranchencodessourcevideoinputswithtemporallyindependentself-attention,andtheintermediatefeaturesareinjectedintothecorrespondingbackboneblocksforstreaming-compatiblecontrol.Moreover,tobridgethediscrepancybetweenthefeaturespacesofbidirectionalandstreamingmodels,weproposeadecoupledtrainingschemethatexplicitlyenforcestheorthogonalitybetweentheoptimizationdirectionsofvideocontrollabilityandmodelcausality.Suchdisentanglementensurescompatibilitybetweenthetwoobjectivesatinferenceandfacilitatessmoothzero-shotknowledgetransferacrossheterogeneousbackbonearchitectures.ExtensiveexperimentsdemonstratethatSVEETachievessuperioreditingqualitywhilemaintainingreal-timeperformance,attaining15FPSonasingleH100GPU17withoutanyauxiliaryaccelerationtechniques.Codesareavailableathttps://github.com/YujiaHu1109/SVEET.

View arXiv pageView PDFGitHub9Add to collection

Get this paper in your agent:

hf papers read 2609\.24788

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.24788 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.24788 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.24788 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles