FlowTool: Controlling Tool Parameter in Image Retouching via Flow Matching

Hugging Face Daily Papers Papers
flow-matching image-editing tool-parameter-control diffusion-transformer multimodal-understanding efficiency-improvement

Summary

FlowTool is a framework that models tool-based image editing as a flow matching problem, achieving superior performance and efficiency compared to autoregressive multimodal models.

Tool-based image editing (image retouching) is commonly formulated with autoregressive multimodal large language models (MLLMs) that sequentially generate reasoning, tool selections, and parameter values. In this work, we present a novel approach to tool-based image editing by framing the task as a flow matching problem. We introduce FlowTool, a framework that directly models the distribution of high-quality tool parameters conditioned on the input image and user instruction using conditional rectified flow. FlowTool combines a vision-language model backbone for multimodal understanding with a Diffusion Transformer parameter generator that transforms Gaussian noise into an editing plan. We train FlowTool with a two-stage supervised flow-matching curriculum, followed by reward-based post-training. Across MMArt-Bench, FlowTool-Eval, ArtEdit-Bench, and MIT-Adobe5K, FlowTool achieves significantly stronger reference-based performance than specialized MLLM editing agents and proprietary MLLMs, while remaining competitive with proprietary models under reference-free evaluation. Moreover, FlowTool significantly improves inference efficiency, reducing latency by at least 50times while requiring nearly 2times less memory than the compared baselines. These results demonstrate that tool-based image editing can be effectively modeled as conditional generation over structured continuous editing parameters, without autoregressive reasoning.
Original Article
View Cached Full Text

Cached at: 09/29/26, 12:10 PM

Paper page - FlowTool: Controlling Tool Parameter in Image Retouching via Flow Matching

Source: https://huggingface.co/papers/2609.35673

Abstract

Tool-basedimageediting(imageretouching)iscommonlyformulatedwithautoregressivemultimodallargelanguagemodels(MLLMs)thatsequentiallygeneratereasoning,toolselections,andparametervalues.Inthiswork,wepresentanovelapproachtotool-basedimageeditingbyframingthetaskasaflowmatchingproblem.WeintroduceFlowTool,aframeworkthatdirectlymodelsthedistributionofhigh-qualitytoolparametersconditionedontheinputimageanduserinstructionusingconditionalrectifiedflow.FlowToolcombinesavision-languagemodelbackboneformultimodalunderstandingwithaDiffusionTransformerparametergeneratorthattransformsGaussiannoiseintoaneditingplan.WetrainFlowToolwithatwo-stagesupervisedflow-matchingcurriculum,followedbyreward-basedpost-training.AcrossMMArt-Bench,FlowTool-Eval,ArtEdit-Bench,andMIT-Adobe5K,FlowToolachievessignificantlystrongerreference-basedperformancethanspecializedMLLMeditingagentsandproprietaryMLLMs,whileremainingcompetitivewithproprietarymodelsunderreference-freeevaluation.Moreover,FlowToolsignificantlyimprovesinferenceefficiency,reducinglatencybyatleast50timeswhilerequiringnearly2timeslessmemorythanthecomparedbaselines.Theseresultsdemonstratethattool-basedimageeditingcanbeeffectivelymodeledasconditionalgenerationoverstructuredcontinuouseditingparameters,withoutautoregressivereasoning.

View arXiv pageView PDFAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.35673 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.35673 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.35673 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

CAT-Flow: Curvature-Adaptive sTeps for Flow Matching

arXiv cs.LG

This paper proposes CAT-OV and CAT-OT, two lightweight, training-free algorithms that adapt step-sizes in Flow Matching sampling based on curvature, improving image quality and reducing generation steps by up to 40%.

Energy-Guided Flow Matching

Hugging Face Daily Papers

Energy-Guided Flow Matching improves generative image quality by using a moving endpoint and adaptive scheduling, achieving state-of-the-art FID scores with reduced training cost.

Follow the Mean: Reference-Guided Flow Matching

Hugging Face Daily Papers

This paper introduces a method for controllable generation in flow matching by adjusting the conditional endpoint mean using a reference set, offering both training-free and semi-parametric guidance for style and content control.