Compositional Text-to-Image Generation Via Region-aware Bimodal Direct Preference Optimization

Hugging Face Daily Papers Papers
text-to-image compositional-generation preference-optimization region-aware diffusion-models fine-tuning

Summary

This paper introduces BiDPO, a framework that enhances text-to-image models for complex compositional prompts through preference-based fine-tuning and region-level guidance, achieving state-of-the-art results on compositional fidelity benchmarks.

Despite the rapid progress of text-to-image (T2I) models, generating images that accurately reflect complex compositional prompts (covering attribute bindings, object relationships, counting) still remains challenging. To address this, we propose BiDPO, a framework to enhance T2I model's capability of compositional text-to-image generation. We begin by introducing an carefully designed pipeline to construct a large-scale preference dataset, BiComp, with strictly quality control. Then, we extend Diffusion DPO to jointly optimize image and text preferences, which is shown to greatly effective in improving the models to follow complex text prompt in generation. To further enhance the models for fine-grained alignment, we employ a region-level guidance method to focus on regions relevant to compositional concepts. Experimental results demonstrate that our BiDPO substantially improves compositional fidelity, consistently outperforming prior methods across multiple benchmarks. Our approach highlights the potential of preference-based fine-tuning for complex text-to-image tasks, offering a flexible and scalable alternative to existing techniques.
Original Article
View Cached Full Text

Cached at: 06/02/26, 03:23 AM

Paper page - Compositional Text-to-Image Generation Via Region-aware Bimodal Direct Preference Optimization

Source: https://huggingface.co/papers/2605.28615

Abstract

BiDPO enhances text-to-image models for complex compositional prompts through preference-based fine-tuning and region-level guidance.

Despite the rapid progress of text-to-image (T2I) models, generating images that accurately reflect complexcompositional prompts(covering attribute bindings, object relationships, counting) still remains challenging. To address this, we propose BiDPO, a framework to enhance T2I model’s capability of compositional text-to-image generation. We begin by introducing an carefully designed pipeline to construct a large-scalepreference dataset,BiComp, with strictly quality control. Then, we extendDiffusion DPOto jointly optimize image and text preferences, which is shown to greatly effective in improving the models to follow complex text prompt in generation. To further enhance the models for fine-grained alignment, we employ aregion-level guidancemethod to focus on regions relevant to compositional concepts. Experimental results demonstrate that our BiDPO substantially improvescompositional fidelity, consistently outperforming prior methods across multiple benchmarks. Our approach highlights the potential ofpreference-based fine-tuningfor complex text-to-image tasks, offering a flexible and scalable alternative to existing techniques.

View arXiv pageView PDFGitHub2Add to collection

Get this paper in your agent:

hf papers read 2605\.28615

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### anzeameol/sdxl-BiDPO Updated4 days ago

Datasets citing this paper1

#### anzeameol/BiComp Viewer• Updated4 days ago • 57.5k • 939

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.28615 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Direct Preference Optimization Beyond Chatbots

Hugging Face Blog

Direct Preference Optimization (DPO) is applied to OCR tasks beyond chatbots, showing significant reduction in text degeneration across multiple model families, with an average reduction of 59.4%.

GroupDPO: Memory efficient Group-wise Direct Preference Optimization

arXiv cs.CL

GroupDPO introduces a memory-efficient algorithm for group-wise direct preference optimization that leverages multiple candidate responses per prompt while reducing peak memory usage through decoupled backpropagation. The method demonstrates consistent improvements over standard DPO across offline and online alignment settings.

$\xi$-DPO: Direct Preference Optimization via Ratio Reward Margin

arXiv cs.LG

This paper introduces xi-DPO, a novel preference optimization method that reformulates the objective to minimize distance to optimal ratio reward margins, addressing hyperparameter tuning challenges in SimPO. Experimental results show that xi-DPO outperforms existing methods on open benchmarks.