StableI2I: Spotting Unintended Changes in Image-to-Image Transition

Hugging Face Daily Papers Papers
image-to-image evaluation-framework computer-vision content-fidelity benchmark

Summary

This paper introduces StableI2I, a reference-free evaluation framework for assessing content fidelity and consistency in image-to-image generation tasks. It also presents StableI2I-Bench, a benchmark for evaluating multi-modal language models on these assessment tasks.

In most real-world image-to-image (I2I) scenarios, existing evaluations primarily focus on instruction following and the perceptual quality or aesthetics of the generated images. However, they largely fail to assess whether the output image preserves the semantic correspondence and spatial structure of the input image. To address this limitation, we propose StableI2I, a unified and dynamic evaluation framework that explicitly measures content fidelity and pre--post consistency across a wide range of I2I tasks without requiring reference images, including image editing and image restoration. In addition, we construct StableI2I-Bench, a benchmark designed to systematically evaluate the accuracy of MLLMs on such fidelity and consistency assessment tasks. Extensive experimental results demonstrate that StableI2I provides accurate, fine-grained, and interpretable evaluations of content fidelity and consistency, with strong correlations to human subjective judgments. Our framework serves as a practical and reliable evaluation tool for diagnosing content consistency and benchmarking model performance in real-world I2I systems.
Original Article
View Cached Full Text

Cached at: 05/08/26, 08:14 AM

Paper page - StableI2I: Spotting Unintended Changes in Image-to-Image Transition

Source: https://huggingface.co/papers/2605.04453

Abstract

StableI2I is a unified evaluation framework that assesses content fidelity and consistency in image-to-image tasks without requiring reference images, providing accurate and interpretable measurements correlated with human judgments.

In most real-worldimage-to-image(I2I) scenarios, existing evaluations primarily focus on instruction following and the perceptual quality or aesthetics of the generated images. However, they largely fail to assess whether the output image preserves the semantic correspondence and spatial structure of the input image. To address this limitation, we proposeStableI2I, a unified and dynamic evaluation framework that explicitly measurescontent fidelityand pre--post consistency across a wide range of I2I tasks without requiring reference images, includingimage editingandimage restoration. In addition, we constructStableI2I-Bench, a benchmark designed to systematically evaluate the accuracy ofMLLMson such fidelity and consistency assessment tasks. Extensive experimental results demonstrate thatStableI2Iprovides accurate, fine-grained, and interpretable evaluations ofcontent fidelityand consistency, with strong correlations to human subjective judgments. Our framework serves as a practical and reliable evaluation tool for diagnosing content consistency and benchmarking model performance in real-world I2I systems.

View arXiv pageView PDFProject pageGitHub10Add to collection

Get this paper in your agent:

hf papers read 2605\.04453

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper2

#### lijiayangCS/StableI2I_PLUS Image-Text-to-Text• 770k• Updated1 day ago • 45 • 1 #### lijiayangCS/StableI2I Image-Text-to-Text• 770k• Updated1 day ago • 88 • 1

Datasets citing this paper1

#### lijiayangCS/StableI2I_Bench Viewer• Updated1 day ago • 3k • 765 • 1

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.04453 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

i1: A Simple and Fully Open Recipe for Strong Text-to-Image Models

Hugging Face Daily Papers

The paper introduces i1, a 3B-parameter text-to-image diffusion model that achieves competitive performance with leading closed models while being fully open (weights, data, code). It provides insights from 300+ controlled experiments and offers a practical recipe for open research.