FlowMimic: Mask-free Visual Editing and Generation with Pixel-pair Warped Flow Field for Online Video Editing Data Generation and Modality Mimicry

Hugging Face Daily Papers Papers
video-editing image-generation flow-field modality-mimicry data-generation mask-free visual-editing

Summary

FlowMimic presents a method for mask-free visual editing and generation across video and image modalities using pixel-pair warped flow fields, enabling real-time video editing data generation from image editing samples and aligning modality capabilities through mimicry losses.

In line with the prevailing direction of vision research, we explore the integration of both generation and editing capabilities for video and image modalities within a single model. Current approaches to collecting video editing data typically depend on labour-intensive, time-consuming curated procedures--involving object mask annotation, the use of error-introducing pair synthesis via I2V model and ControlNet-like guidance, and VLM-based quality filtering or refinement--and demonstrate limited task scalability. As a result, the diversity of editing tasks remains substantially narrower than that available for image editing models. We develop a pixel-pair temporal warped flow field that can directly generate corresponding video editing samples in real time from image editing samples, and we demonstrate across multiple levels of video editing tasks that a model can learn video editing using only such data. We regard the image modality as a particular form of the video modality. Accordingly, we design a modality mimic generation loss and a modality mimic editing loss to relatively align the capabilities--and thereby the output distributions--of the two modalities through mutual imitation. Moreover, language-based visual editing entails the comprehension of the editing instruction and the reference visual content, the localization of the region corresponding to that instruction within the reference visual contents, and the modification of that region alone. Existing approaches predominantly rely on external aids, such as fine-tuning an additional MLLM or explicitly supplying a mask sequence as auxiliary input during inference. In contrast, we aspire for the model to internalize this capability. To that end, we introduce sense-related tasks--for instance, referring expression segmentation--along with corresponding editing-region-aware latent-level loss and attention-level loss.
Original Article
View Cached Full Text

Cached at: 07/21/26, 10:36 AM

Paper page - FlowMimic: Mask-free Visual Editing and Generation with Pixel-pair Warped Flow Field for Online Video Editing Data Generation and Modality Mimicry

Source: https://huggingface.co/papers/2607.18227

Abstract

Inlinewiththeprevailingdirectionofvisionresearch,weexploretheintegrationofbothgenerationandeditingcapabilitiesforvideoandimagemodalitieswithinasinglemodel.Currentapproachestocollectingvideoeditingdatatypicallydependonlabour-intensive,time-consumingcuratedprocedures--involvingobjectmaskannotation,theuseoferror-introducingpairsynthesisviaI2VmodelandControlNet-likeguidance,andVLM-basedqualityfilteringorrefinement--anddemonstratelimitedtaskscalability.Asaresult,thediversityofeditingtasksremainssubstantiallynarrowerthanthatavailableforimageeditingmodels.Wedevelopapixel-pairtemporalwarpedflowfieldthatcandirectlygeneratecorrespondingvideoeditingsamplesinrealtimefromimageeditingsamples,andwedemonstrateacrossmultiplelevelsofvideoeditingtasksthatamodelcanlearnvideoeditingusingonlysuchdata.Weregardtheimagemodalityasaparticularformofthevideomodality.Accordingly,wedesignamodalitymimicgenerationlossandamodalitymimiceditinglosstorelativelyalignthecapabilities--andtherebytheoutputdistributions--ofthetwomodalitiesthroughmutualimitation.Moreover,language-basedvisualeditingentailsthecomprehensionoftheeditinginstructionandthereferencevisualcontent,thelocalizationoftheregioncorrespondingtothatinstructionwithinthereferencevisualcontents,andthemodificationofthatregionalone.Existingapproachespredominantlyrelyonexternalaids,suchasfine-tuninganadditionalMLLMorexplicitlysupplyingamasksequenceasauxiliaryinputduringinference.Incontrast,weaspireforthemodeltointernalizethiscapability.Tothatend,weintroducesense-relatedtasks--forinstance,referringexpressionsegmentation--alongwithcorrespondingediting-region-awarelatent-levellossandattention-levelloss.

View arXiv pageView PDFAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.18227 in a model README.md to link it from this page.

Datasets citing this paper1

#### FlowMimic/Uncompressed Updatedabout 7 hours ago • 13

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.18227 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Mage-Flow: An Efficient Native-Resolution Foundation Model for Image Generation and Editing

Hugging Face Daily Papers

Mage-Flow is a compact 4B-parameter generative stack for efficient text-to-image generation and instruction-based image editing, featuring a co-designed lightweight tokenizer (Mage-VAE) and a native-resolution multimodal diffusion transformer trained with rectified flow matching. It achieves competitive performance while enabling high-resolution generation at 0.59s on a single A100 GPU.

Masked Language Flow Models

arXiv cs.CL

This paper introduces Masked Language Flow Models (MLFMs), which incorporate masking into flow-based language models to enable continuous flow for conditional generation and allow pretrained Masked Diffusion Models to be converted. The authors propose a novel sampler that alternates continuous denoising with discrete unmasking, demonstrating for the first time that flow-based language models can scale to downstream reasoning and instruction-following tasks.

DanceOPD: On-Policy Generative Field Distillation

Hugging Face Daily Papers

DanceOPD proposes an on-policy generative field distillation framework for flow-matching models that unifies text-to-image generation, local editing, and global editing via capability-specific routing and velocity-based training, improving multi-capability composition while preserving anchor generation quality.