Text-Vision Co-Instructed Image Editing
Summary
A new framework called TV-Edit combines textual instructions and visual prompts for precise image editing, along with a benchmark TV-Edit-Bench for evaluation. The method achieves better spatial control and semantic faithfulness than existing approaches.
View Cached Full Text
Cached at: 06/17/26, 11:37 AM
Paper page - Text-Vision Co-Instructed Image Editing
Source: https://huggingface.co/papers/2606.16767
Abstract
A unified text-visual image editing framework is presented that combines semantic intent from textual instructions with spatial guidance from visual prompts to achieve more precise and faithful image manipulation.
Existing image editing methods can be generally categorized intotextual instruction-basedandvisual prompt-basedones. Textual instructions are semantically expressive, but are limited by the coarse granularity ofspatial controlof the editing results. In contrast, visual prompts such as drag and point can provide precisespatial guidance, but are limited by the inherent ambiguity insemantic intent. To unify the strength of textual and visual prompts, we present Text-Vision Co-Instructed Image Editing, which jointly models textual instructions assemantic intentand sparse visual instructions asspatial guidance, aiming to achieve precise and intent-faithful image manipulation. To this end, we first construct atextual-visual instruction paired datasetwith more than 23K samples derived from dynamic videos, enabling aligned supervision forcross-modal instruction. We then proposeTV-Edit, a Textual-Visual instruction unified Editing framework to contextualize drag or point-based visual instructions with image-text semantics and lift them intosemantic-aware control representationsforpretrained editing backbones. By integratingsemantic intentand spatial constraints,TV-Editleads to more precisespatial control, less instruction ambiguity, and strongerstructural consistencythan text-only or drag-based alternatives. Finally, we establishTV-Edit-Bench, a deliberately designed benchmark to evaluate semantic faithfulness, spatial alignment, and visual consistency with ground-truth references and controlled textual-visual variations for reliable assessment. Our experiments across multiple editing backbones demonstrate thatTV-Editconsistently yields more precise andintent-faithful edits, significantly outperforming state-of-the-art instruction-based and drag-based baselines.
View arXiv pageView PDFProject pageGitHub14Add to collection
Get this paper in your agent:
hf papers read 2606\.16767
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.16767 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.16767 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.16767 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
VGGT-Edit: Feed-forward Native 3D Scene Editing with Residual Field Prediction
VGGT-Edit proposes a feed-forward framework for text-conditioned native 3D scene editing using depth-synchronized text injection and residual field prediction, achieving superior quality and efficiency over 2D-lifting approaches.
Uni-Edit: Intelligent Editing Is A General Task For Unified Model Tuning
Uni-Edit proposes using intelligent image editing as a single general task to simultaneously improve unified multimodal models' understanding, generation, and editing capabilities, with an automated data synthesis pipeline creating complex editing instructions.
TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment
TIPSv2 introduces enhanced vision-language pretraining techniques including patch-level distillation, an upgraded masked image objective (iBOT++), and improved caption sampling strategies to achieve superior dense patch-text alignment. The resulting family of image-text encoder models demonstrates strong performance across 9 tasks and 20 datasets.
HP-Edit: A Human-Preference Post-Training Framework for Image Editing
HP-Edit introduces a post-training framework that aligns diffusion-based image editing models with human preferences via RLHF, using a new 50K real-world dataset and an automatic VLM-based evaluator.
IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generation
IV-CoT decomposes visual conditioning into structural and semantic cascades for improved structure-aware image generation, using training-only sketch supervision to guide structural queries. It achieves state-of-the-art results on GenEval and T2I-CompBench.