Paper page - CanvasAgent: Enabling Complex Image Creation and Editing via Visual Tool Orchestration

Source: https://huggingface.co/papers/2607.05465 Published on Jul 6

·

Submitted byhttps://huggingface.co/cswhjiang

whon Jul 8

Abstract

A large-scale multimodal tool-use dataset and agent are presented for complex image creation workflows that orchestrate multiple visual tools through multi-turn interactions and hybrid reward optimization.

Complex image creationand editing often require more than a single generation or editing model. A user request may involve synthesizing images, localizing objects, segmenting regions, editing selected content, compositing intermediate assets, reading text, and enhancing the final result. Such tasks shift multimodal agents from perception-augmented reasoning to manipulation-centered visual creation, where tools must actively transform visual states rather than merely inspect them. However, existing multimodal tool-use agents are mostly optimized for perception, search, or domain-specific editing, and lack large-scale supervision for executable image-creation trajectories. In this paper, we introduce CanvasCraft, a large-scalemultimodal tool-use datasetforcomplex image creationand editing, and CanvasAgent, atool-augmented multimodal agentthat learns to orchestrate heterogeneousvisual toolsthroughmulti-turn interaction. CanvasCraft contains 140K fully annotatedexecutable trajectoriesand 10K RL task specifications. CanvasAgent is first trained with SFT to learn executable reasoning-action trajectories, and is then optimized withGRPOusing ahybrid rewardthat combines outcome- and process-level signals. During rollout, CanvasAgent inspects intermediate results, tracks visual assets, and adapts tool decisions to the evolving visual state. Experiments evaluate both final image quality and trajectory behavior, demonstrating the effectiveness of CanvasAgent and the proposed dataset for complex multi-tool image creation workflows.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2607\.05465

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.05465 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.05465 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.05465 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.