SynCred-Bench: Benchmarking Synthetic Credibility in AI-Generated Visual Misinformation
Summary
Introduces SynCred-Bench, a benchmark of 600 AI-generated misinformation images across six credible-form categories, showing that existing detectors (including MLLMs, open-source AIGC detectors, and commercial APIs) perform poorly, with human annotators also struggling.
View Cached Full Text
Cached at: 06/04/26, 03:42 AM
Paper page - SynCred-Bench: Benchmarking Synthetic Credibility in AI-Generated Visual Misinformation
Source: https://huggingface.co/papers/2606.03348
Abstract
AI-generated images with realistic text and layouts pose a significant misinformation threat requiring new detection benchmarks and methods beyond surface-level credibility assessment.
Recentgenerative modelscan now produce visual artifacts with realistic embedded text and layouts, creating a newmisinformationthreat:synthetic credibility. We introduceSYNCRED-Bench, a benchmark of 600 AI-generatedmisinformationimages balanced across six credible-form categories and seven fine-grained circulation styles, together withFP450, a real-image negative set for measuring false positives. Extensive evaluation shows that existing systems remain unreliable: under a 5%false-positive-rateconstraint, 15MLLMsachieve only 10.5%true positive rate(TPR), open-sourceAIGC detectorsachieve less than 5%, andcommercial APIsreach 57.6%. Human annotators also struggled to identifysynthetic credibility, reaching only 63% TPR. These findings establishsynthetic credibilityas a severe and underexplored visualmisinformationchallenge, and provide a benchmark for developing detectors that reason beyond superficial credibility cues.
View arXiv pageView PDFAdd to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.03348 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.03348 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.03348 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Can We Defend Against AI-Generated Video Attacks on Real-World Crisis Events? A Systematic Evaluation of Detectors, Generators and Social Dissemination
The paper introduces RA-Bench, a new benchmark for evaluating AI-generated video detection in real-world crisis events, demonstrating that current detectors fail to generalize and become less reliable during social dissemination.
Artifact-Bench: Evaluating MLLMs on Detecting and Assessing the Artifacts of AI-Generated Videos
Artifact-Bench is a comprehensive benchmark that evaluates multimodal large language models on detecting and analyzing artifacts in AI-generated videos, revealing significant limitations and misalignment with human perception.
DF26: We Cannot Tell Fake From Real Anymore
The paper introduces DF26, a benchmark for detecting AI-generated public-speaking videos, showing that both humans and current detectors perform near chance, underscoring the need for improved robustness to generative model distribution shifts.
Every AI Visibility Tool Is Lying to You
This article critically examines the accuracy of AI visibility tools that claim to measure brand presence in generative AI responses, arguing that they provide false precision due to nondeterminism, personalization, and scraping biases. It calls for transparency in methodology and warns against treating opaque dashboards as stable truth.
Is this chart lying to me? Automating the detection of misleading visualizations
This paper introduces Misviz, a benchmark dataset of 2,604 real-world visualizations and 57,665 synthetic ones annotated with 12 types of misleading design violations, enabling automated detection of deceptive charts. The work evaluates state-of-the-art multimodal LLMs and rule-based systems on this challenging task, addressing the gap in resources for training AI models to combat data visualization misinformation.