Are Text-to-Image Models Inductivist Turkeys? A Counterfactual Benchmark for Causal Reasoning
Summary
This paper introduces CF-World, a counterfactual benchmark to evaluate whether text-to-image models rely on causal reasoning or mere pattern matching. Experiments show all models degrade sharply in counterfactual settings, suggesting their understanding is limited to tightly coupled visual-textual patterns rather than genuine causal reasoning.
View Cached Full Text
Cached at: 06/24/26, 05:46 AM
Paper page - Are Text-to-Image Models Inductivist Turkeys? A Counterfactual Benchmark for Causal Reasoning
Source: https://huggingface.co/papers/2606.24548 Authors:
,
,
,
,
,
,
,
,
,
Abstract
Text-to-image models fail to generate counterfactual scenes because they rely on tightly coupled visual-textual patterns rather than causal reasoning, demonstrating limited understanding beyond pattern matching.
Text-to-image (T2I) generation models have achieved remarkable progress in producing visually realistic images from natural language prompts. Yet it remains unclear whether their success reflects genuinecausal understandingor sophisticatedpattern matchingovervisual-textual correlations. Inspired by Russell’s inductivist turkey, we introduce Counterfactual-World (CF-World), acounterfactual benchmarkdesigned to investigate whether text-to-image models can generate images under rules that systematically contradict real-world priors. CF-World organizes each scenario into three progressive levels: factual generation under ordinary world knowledge, explicit counterfactual generation with direct visual instructions, and implicit counterfactual generation requiring causal deduction from altered rules. We evaluate both open-source and closed-source T2I models using aVision Language Model(VLM)-based evaluator (CF-Eval). Furthermore, we introduce two metrics:Prior Resistance Rate(PRR), which measures a model’s ability to overcome entrenched real-world priors, andReasoning Retention Rate(RRR), which assesses whether models can maintain reasoning-dependent counterfactual generation without explicit visual cues. Experiments show that all models exhibit sharp degradation from factual to counterfactual settings. Further analyses suggest that these failures arise because current T2I models encode world knowledge and visual appearances as tightly coupled patterns. Consequently, their heavy reliance on frequent visual co-occurrences within the training data forces them to default to familiar commonsense priors when tasked with rendering counterfactual worlds.
View arXiv pageView PDFProject pageAdd to collection
Get this paper in your agent:
hf papers read 2606\.24548
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.24548 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.24548 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.24548 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Evidence-RL: Towards Evidence-intensive Visual Reasoning
This paper introduces Counterfactual Evidence Disentanglement (CED), a training-time method that makes vision-language models rely on concrete image evidence rather than language priors or shortcuts, improving visual reasoning grounding across benchmarks.
YoCausal: How Far is Video Generation from World Model? A Causality Perspective
This paper introduces YoCausal, a benchmark based on the Violation of Expectation paradigm from cognitive science, to evaluate whether video diffusion models truly understand causality or merely overfit to temporal patterns. Evaluation of 13 state-of-the-art models reveals a significant gap compared to human-level causal cognition.
Vernier: Probing Representational Misalignment Behind Lexical Gaps in Causal Reasoning
This paper investigates why instruction-tuned language models give different answers to causal reasoning questions when variable names are replaced with placeholders, finding that the issue stems from representational misalignment rather than information loss. The authors introduce Vernier, a method using paired-view weight updates and mechanism inspection to reveal that answer-relevant content is still present in the placeholder view but misaligned.
CausalDS: Benchmarking Causal Reasoning in Data-Science Agents
Introduces CausalDS, a benchmark for evaluating causal reasoning in LLM-based data science agents, using synthetic structural causal models and natural language stories to test associational, interventional, and counterfactual reasoning along with tool use and abstention.
Evidence-Type Competition: When Can Interventional Data Teach Language Models Causal Direction?
This paper tests whether increasing interventional data in pretraining improves LLMs' causal direction reasoning, using controlled Simpson's-paradox worlds. It finds that the training mixture does not govern interventional evidence use; instead the evidence type in the inference-time context is the decisive factor.