FPCO-Dialog: A Multi-Turn False-Premise Benchmark for Correction and Cooperation in Vision-Language Models

# FPCO-Dialog: A Multi-Turn False-Premise Benchmark for Correction and Cooperation in Vision-Language Models
Source: [https://arxiv.org/html/2609.03331](https://arxiv.org/html/2609.03331)
Jiayuan MaYuqi LuAffiliation:Harbin Institute of Technology, Shenzhen, ChinaEmail:[jingli\.phd@hotmail\.com](mailto:)Weiyang GuoAffiliation:Harbin Institute of Technology, Shenzhen, ChinaChenrui WangAffiliation:Harbin Institute of Technology, Shenzhen, ChinaJunyi ShuAffiliation:Harbin Institute of Technology, Shenzhen, ChinaXuebo LiuAffiliation:Harbin Institute of Technology, Shenzhen, ChinaMin ZhangAffiliation:Harbin Institute of Technology, Shenzhen, ChinaJing LiAffiliation:Harbin Institute of Technology, Shenzhen, China

###### Abstract

Vision\-language models \(VLMs\) are increasingly deployed in multi\-turn settings where users may describe visual content with incorrect assumptions\. Yet existing evaluations rarely isolate how models respond when the same visually grounded false premise persists across dialogue turns\. We introduce FPCO\-Dialog, a benchmark for evaluating correction and cooperation behavior in VLMs under repeated false premises\. FPCO\-Dialog contains 1,080 images and 10,800 question turns, stratified by visual complexity, object category, and false\-premise class, and uses a 10\-turn protocol in which a correct dialogue prefix is followed by repeated false\-premise referring expressions\. We evaluate 20 commercial and open\-source VLMs with a model\-agnostic protocol and CorrTP@K, a correction\-rate metric over false\-premise turns, scored by two independent detectors\. FPCO\-Dialog reveals substantial and persistent cross\-model differences in aggregate correction tendency, model\-specific turn\-wise dynamics, and systematic variation across false\-premise types under the benchmark’s substitution distribution\. The dataset, evaluation protocol, model outputs, detector labels, and code are available at[https://github\.com/lab\-klc/FPCO\-Dialog](https://github.com/lab-klc/FPCO-Dialog)\.††footnotetext:Corresponding author\.

![Refer to caption](https://arxiv.org/html/2609.03331v1/motivation_ver.png)Figure 1:Correction vs\. Cooperation\.Under the same visually grounded false premise, VLMs may either cooperate with the user’s main question or explicitly correct the premise before or while answering\.## 1Introduction

Vision\-language models \(VLMs\) are increasingly evaluated in settings that move beyond label prediction or short\-answer visual question answering, including visually grounded interactive dialogue\([Das et al\., 2017](https://arxiv.org/html/2609.03331#bib.bib10);[Cao et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib11);[Lee et al\., 2025](https://arxiv.org/html/2609.03331#bib.bib12)\)\. Recent VLM benchmarks test broad multimodal competence, including integrated perception and reasoning, discipline\-specific problem solving, hallucination robustness, and preference\-aligned open\-ended response quality\([Yu et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib4);[Yue et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib5);[Guan et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib6);[Lu et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib7)\)\. Yet most evaluations either ask independent questions or use naturally collected dialogue histories, making it difficult to isolate how a model responds when a user repeatedly describes visible content using a premise that conflicts with the image\([Das et al\., 2017](https://arxiv.org/html/2609.03331#bib.bib10);[Cao et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib11);[Lee et al\., 2025](https://arxiv.org/html/2609.03331#bib.bib12)\)\. Figure[1](https://arxiv.org/html/2609.03331#S0.F1)illustrates this ambiguity: when a user refers to a train as a bus, a VLM may either cooperate with the user’s main question or first correct the false premise\. FPCO\-Dialog is designed to measure this correction–cooperation behavior under a controlled repeated\-false\-premise protocol\. A single\-turn response may reveal whether a model notices one inconsistency, but repeated turns are needed to test whether correction or cooperation reflects a stable interaction strategy\.

This setting is important because referring expressions do not merely identify an object; they also introduce presuppositions about the shared visual context, such as the identity, attribute, or location of a target\([Lewis, 1979](https://arxiv.org/html/2609.03331#bib.bib13);[Stalnaker, 2002](https://arxiv.org/html/2609.03331#bib.bib14);[Simons, 2003](https://arxiv.org/html/2609.03331#bib.bib15)\)\. In dialogue, such presuppositions may be accommodated for cooperation or explicitly repaired when incompatible with available evidence\. Existing QA and VQA work has studied unanswerable questions, abstention, object hallucination, and false\-premise questions, but these lines often frame problems as detecting unsupported answers rather than measuring the balance between correction and cooperation in visually grounded dialogue\([Rajpurkar et al\., 2018](https://arxiv.org/html/2609.03331#bib.bib16);[Gurari et al\., 2018](https://arxiv.org/html/2609.03331#bib.bib17);[Whitehead et al\., 2022](https://arxiv.org/html/2609.03331#bib.bib18);[Li et al\., 2023](https://arxiv.org/html/2609.03331#bib.bib20);[Hu et al\., 2023](https://arxiv.org/html/2609.03331#bib.bib19)\)\. Recent visual\-dialogue and hallucination benchmarks further show that dialogue history can affect VLM reliability, but they do not impose a controlled repeated same\-false\-premise protocol that separates premise type from image complexity and object category\([Cao et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib11);[Guan et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib6);[Lee et al\., 2025](https://arxiv.org/html/2609.03331#bib.bib12)\)\.

We introduce FPCO\-Dialog, a multi\-turn false\-premise benchmark for studying correction and cooperation behavior in VLMs\. Each instance is built from an image selected from MS COCO and Open Images\([Lin et al\., 2014](https://arxiv.org/html/2609.03331#bib.bib1);[Kuznetsova et al\., 2020](https://arxiv.org/html/2609.03331#bib.bib2)\)and paired with a ten\-turn visual dialogue: an initial premise\-correct prefix is followed by repeated turns using the same false\-premise referring expression\. The benchmark contains 1,080 images stratified by visual complexity, object category, and false\-premise class, covering identity, attribute, and location errors\. This design isolates how VLMs respond to persistent visually grounded false premises while keeping the dialogue protocol fixed and model\-agnostic\.

Our main contributions are as follows:

- •We introduce FPCO\-Dialog, a controlled and stratified multi\-turn benchmark for repeated visually grounded false premises, covering 1,080 images across visual complexity, object category, and false\-premise class\.
- •We propose a model\-agnostic evaluation framework for measuring correction and cooperation behavior, including a fixed dialogue protocol, dual\-detector scoring, a new indicator CorrTP@K, model outputs, detector labels, and evaluation code\.
- •We conduct a 20\-model empirical study with quantitative and qualitative analyses which show substantial differences across model families and systematic variation across false\-premise types\.

![Refer to caption](https://arxiv.org/html/2609.03331v1/dataset.png)Figure 2:Dataset Construction and Structure\.a\) FPCO\-Dialog is built from COCO and Open Images through manual selection, annotation, categorization, question generation, and false\-premise rewriting\. b\) The dataset contains 1,080 images stratified by visual complexity, object category, and false\-premise class\. c\) Each dialogue contains three premise\-correct turns followed by repeated false\-premise turns; the shown example illustrates all three false\-premise classes for compact visualization\.
## 2Related Work

#### Vision\-language evaluation and visual dialogue\.

Early visual question answering and visual dialogue benchmarks established image\-conditioned question answering and dialogue grounding as core evaluation settings\([Antol et al\., 2015](https://arxiv.org/html/2609.03331#bib.bib21);[Das et al\., 2017](https://arxiv.org/html/2609.03331#bib.bib10)\)\. Recent VLM benchmarks extend this line to integrated perception and reasoning, expert\-level multimodal problem solving, hallucination diagnosis, and preference\-based open\-ended evaluation\([Liu et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib3);[Yu et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib4);[Yue et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib5);[Guan et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib6);[Lu et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib7)\)\. Evaluation toolkits further provide reproducible pipelines for comparing large multimodal models across benchmarks\([Duan et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib8)\)\. Recent multi\-turn benchmarks study dialogue history, hallucination, and complex conversational goals in visually grounded interaction\([Cao et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib11);[Lee et al\., 2025](https://arxiv.org/html/2609.03331#bib.bib12)\)\. FPCO\-Dialog is complementary to these efforts: rather than maximizing task diversity, it fixes the dialogue protocol and systematically varies false\-premise type, visual complexity, and object category to isolate correction behavior\.

#### False premises, presupposition, and visually grounded inconsistency\.

Our task is related to work on presupposition and common ground, where referring expressions can introduce assumptions that interlocutors may accommodate or repair depending on the conversational state\([Lewis, 1979](https://arxiv.org/html/2609.03331#bib.bib13);[Stalnaker, 2002](https://arxiv.org/html/2609.03331#bib.bib14);[Simons, 2003](https://arxiv.org/html/2609.03331#bib.bib15)\)\. In NLP and VQA, related phenomena have been studied through unanswerable questions, abstention\-oriented answering, naturally unanswerable visual questions, and false\-premise question answering\([Rajpurkar et al\., 2018](https://arxiv.org/html/2609.03331#bib.bib16);[Gurari et al\., 2018](https://arxiv.org/html/2609.03331#bib.bib17);[Whitehead et al\., 2022](https://arxiv.org/html/2609.03331#bib.bib18);[Hu et al\., 2023](https://arxiv.org/html/2609.03331#bib.bib19)\)\. VLM hallucination benchmarks examine whether generated responses remain grounded in the image, especially when objects or visual facts are unsupported\([Li et al\., 2023](https://arxiv.org/html/2609.03331#bib.bib20);[Guan et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib6)\)\. FPCO\-Dialog differs in the direction and dynamics of inconsistency: the false content is supplied by the user, repeated across dialogue turns, and evaluated by whether the model corrects the premise or cooperates with the user’s main question\. FPCO\-Dialog is also related to work on sycophancy, but we do not equate cooperation with sycophancy, since non\-correction may reflect pragmatic accommodation or failure to detect the inconsistency rather than deference to the user\([Sharma et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib37);[Hong et al\., 2025](https://arxiv.org/html/2609.03331#bib.bib38);[Pi et al\., 2025](https://arxiv.org/html/2609.03331#bib.bib39)\)\.

![Refer to caption](https://arxiv.org/html/2609.03331v1/sample.png)Figure 3:Examples of Dataset Stratification and False\-premise Modifications\.a\) Representative FPCO\-Dialog images across visual complexity and object category\. Rows correspond to single\-target clean, multi\-target separated, and multi\-target overlapping settings; columns correspond to person, vehicle, animal, and food categories\. b\) Examples of referring\-expression modifications for the three false\-premise classes: identity, attribute, and location\.

## 3FPCO\-Dialog Dataset

FPCO\-Dialog measures how VLMs respond to repeated visually grounded false premises\. Each dialogue centers on a target entity in an image\. Premise\-correct turns refer to the target with a correct description, while false\-premise turns use a modified referring expression\. Figure[2](https://arxiv.org/html/2609.03331#S1.F2)summarizes construction, stratification, and a representative dialogue example\.

### 3\.1Task and Dialogue Protocol

Each instance contains an image, a target entity, a correct target description, a modified false\-premise target description, and a ten\-turn question sequence\. At each turn, a VLM receives the image, the current question, and the preceding dialogue context, then generates a free\-form response\.

The first three turns use the correct target description and form a premise\-correct dialogue prefix\. Turns 4–10 repeatedly use the same modified false\-premise referring expression\. This schedule isolates whether a model maintains, changes, or suppresses correction behavior when the same visual inconsistency persists across turns\. On false\-premise turns, a model may explicitly correct the premise, for example by stating that the referred object is absent or that the visual description is wrong\. Alternatively, it may cooperate with the user’s main question and answer without addressing the inconsistency\. FPCO\-Dialog measures this correction–cooperation distinction under a controlled repeated\-false\-premise protocol\.

### 3\.2Construction and Stratification

We build FPCO\-Dialog from images selected from MS COCO and Open Images\([Lin et al\., 2014](https://arxiv.org/html/2609.03331#bib.bib1);[Kuznetsova et al\., 2020](https://arxiv.org/html/2609.03331#bib.bib2)\)\. Images are manually selected and organized to support controlled comparisons across visual complexity, object category, and false\-premise class\. The authors define the benchmark design, stratification scheme, and construction constraints, while LLMs assist with question generation and false\-premise rewriting\. All generated instances are manually reviewed by the authors for visual grounding, target consistency, naturalness, and compliance with the single\-error constraint\. We do not rely on the original dataset annotations for benchmark labels or evaluation; target descriptions, question sequences, false\-premise modifications, metadata, model outputs, and detector labels are generated or curated as part of FPCO\-Dialog\.

The dataset contains 1,080 images stratified along three axes\. Visual complexity \(Figure[3](https://arxiv.org/html/2609.03331#S2.F3)a\) has three levels: single\-target clean, multi\-target separated, and multi\-target overlapping\. Object category has four levels: person, vehicle, animal, and food\. False\-premise class has three levels: identity, attribute, and location\. For each visual\-complexity–category combination, we include 90 images, evenly divided across the three false\-premise classes, yielding3×4×3×30=1,0803\\times 4\\times 3\\times 30=1\{,\}080images\.

False\-premise classes are defined by how the referring expression conflicts with the image\. As shown in Figure[3](https://arxiv.org/html/2609.03331#S2.F3)b,Identityfalse premises replace the target identity with an incompatible alternative, such as referring to a train as a bus\.Attributefalse premises modify a visible property of the target, such as color, material, or expression\.Locationfalse premises modify the target’s spatial description or relation, such as replacing “right” with “left”\. These examples are illustrative rather than fixed replacement mappings\. For each instance, the substitution is generated independently under a constrained minimal\-edit policy: we change one word when possible, otherwise one short local phrase, preserve the remaining description, and introduce only one error of the assigned false\-premise class\.

For each image, we first define one correct target description and generate ten premise\-correct questions that contain this description\. We then create one modified target description according to the image’s assigned false\-premise class and rewrite turns 4–10 by replacing the correct referring expression with the modified one\. Identity substitutions use a clearly incompatible label, attribute substitutions change one visible property, and location substitutions change the spatial relation of the whole target\. Thus, each image has exactly one target description, one modified target description, and one false\-premise class\. The example in Figure[2](https://arxiv.org/html/2609.03331#S1.F2)shows all three false\-premise classes only for compact visualization; in the actual benchmark, each dialogue contains a single repeated false\-premise class\.

### 3\.3Release and license

We release the dataset, evaluation protocol, model outputs, detector labels, and code\. Because the images are selected from MS COCO and Open Images, FPCO\-Dialog is released as a mixed\-license benchmark: source images remain subject to their original licenses, while author\-generated components such as metadata, questions, false\-premise modifications, evaluation scripts, and detector labels are released for research use\.

## 4Experiment Setup

![Refer to caption](https://arxiv.org/html/2609.03331v1/benchmark.png)Figure 4:Benchmark Procedure\.VLMs answer questions conditioned on image and dialogue context\. Responses are scored by two detectors, and CorrTP@K measures correction behavior on false\-premise turns\.### 4\.1Evaluated Models

Figure 5:Evaluated Models\.Twenty commercial/API\-based and open\-source VLMs are evaluated from five model families\.We evaluate 20 commercial/API\-based and open\-source VLMs from five model families: Gemini\([Google DeepMind, 2025a](https://arxiv.org/html/2609.03331#bib.bib24);[Google DeepMind, 2025b](https://arxiv.org/html/2609.03331#bib.bib25);[Google DeepMind, 2026](https://arxiv.org/html/2609.03331#bib.bib26)\), GPT\([OpenAI, 2024](https://arxiv.org/html/2609.03331#bib.bib27);[OpenAI, 2025a](https://arxiv.org/html/2609.03331#bib.bib29);[OpenAI, 2025b](https://arxiv.org/html/2609.03331#bib.bib28);[OpenAI, 2026](https://arxiv.org/html/2609.03331#bib.bib30)\), Qwen\([Bai et al\., 2023](https://arxiv.org/html/2609.03331#bib.bib31);[Bai et al\., 2025](https://arxiv.org/html/2609.03331#bib.bib9);[Qwen Team, 2026a](https://arxiv.org/html/2609.03331#bib.bib32);[Qwen Team, 2026b](https://arxiv.org/html/2609.03331#bib.bib33)\), InternVL\([Chen et al\., 2025](https://arxiv.org/html/2609.03331#bib.bib34)\), and LLaVA\([Liu et al\., 2023](https://arxiv.org/html/2609.03331#bib.bib36);[Li et al\., 2025](https://arxiv.org/html/2609.03331#bib.bib35)\)\. As summarized in Figure[5](https://arxiv.org/html/2609.03331#S4.F5), the model set covers frontier proprietary systems and open\-source models across different scales\. Each model produces one response for every question turn, yielding 10,800 responses per model and 216,000 model responses in total\. We do not train or fine\-tune any model; all experiments are inference\-only evaluations\. Model scales are reported where publicly available through model names or official documentation, while exact parameter counts and backend compute for proprietary API models are not publicly disclosed\.

### 4\.2Benchmarking Procedure

Figure[4](https://arxiv.org/html/2609.03331#S4.F4)summarizes the evaluation procedure\. For each image, a model is evaluated on the ten\-turn dialogue defined by FPCO\-Dialog\. At each turn, the model receives the image, the current user question, and the preceding dialogue context, then generates a free\-form natural\-language response\. Responses are generated sequentially so that later turns include the earlier context in the dialogue history\.

All models are evaluated with the same benchmark instances, turn order, and false\-premise schedule\. Models are not given the false\-premise class, the correct target description, the modified target description, or detector labels\. This prevents benchmark\-side annotations from influencing response generation during inference\. This setup helps isolate model behavior rather than differences in benchmark access\. The protocol is therefore model\-agnostic: any VLM that supports image\-conditioned multi\-turn responses can be evaluated by running the same dialogue and applying the same scoring pipeline\. API\-specific and local\-inference scripts only adapt input formatting to each model interface; benchmark content and scoring are held fixed\. For reproducibility, all model responses are generated with fixed inference settings within each model interface\. We use deterministic decoding where supported, keep the maximum generation length fixed within each interface, and report detailed API, local inference, detector, and aggregation settings in Appendix[I](https://arxiv.org/html/2609.03331#A9)\. The same benchmark content and dialogue order are used for all models\.

### 4\.3Dual\-detector Scoring

Because model responses are open\-ended, exact\-match scoring is not suitable\. Following LLM\-as\-judge and evaluator\-model work\([Zheng et al\., 2023](https://arxiv.org/html/2609.03331#bib.bib22);[Kim et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib23)\), as well as multimodal evaluation frameworks\([Duan et al\., 2024](https://arxiv.org/html/2609.03331#bib.bib8)\), FPCO\-Dialog uses detector\-based scoring\. Unlike general\-purpose preference or quality judging, our detector task is restricted to a narrow behavioral criterion: whether the response corrects the false premise\.

We use two independent detector models, GPT\-5\.4 and Gemini\-3\.1\-Pro\-Preview\. Each detector receives, for each turn, the shared false\-premise class, a flag indicating whether the turn contains a false premise, the premise\-correct question, the corresponding false\-premise question when applicable, and the model response\. A response is labeled as a correction if it explicitly identifies the premise as inconsistent or clearly repairs the injected premise in its answer, such as stating that the referred object is absent, that the object is not the described entity, that a visible attribute or location is incorrect, or using the correct premise instead of the modified false premise\. A response is not labeled as a correction if it simply answers the user’s main question while accepting or ignoring the false premise\. The main reported label is the arithmetic mean of the two binary detector labels\.

Figure 6:Main Correction Behavior in FPCO\-Dialog\.a\) Representative models exhibit stable differences in average CorrTP@K across repeated false\-premise turns\. b\) Correction behavior differs strongly by false\-premise type, with identity errors corrected most often and location errors most often accommodated\.Figure 7:Effects of Visual Complexity and Its Interaction with False\-premise Type\.a\) Average CorrTP@10 shows modest overall variation across visual complexity settings\. b\) Crossing visual complexity with false\-premise type reveals clearer interaction patterns than the complexity\-only view\.
### 4\.4Metrics

We report CorrTP@K, a cumulative turn\-indexed correction metric over false\-premise turns up to dialogue positionKK\. For each cutoff turnKK, we collect all response items from turns up toKKand re\-index them asi=1,…,Ni=1,\\ldots,N\. Letei=1e\_\{i\}=1indicate that the selected response item contains a false premise, and letci\(K\)=1c\_\{i\}^\{\(K\)\}=1indicate that the response is labeled as a correction by the detector\. CorrTP@K is defined as:

CorrTP​@​K=∑i=1N𝕀⁡\(ei=1∧ci\(K\)=1\)∑i=1N𝕀⁡\(ei=1\)\.\\mathrm\{CorrTP@K\}=\\frac\{\\sum\_\{i=1\}^\{N\}\\mathbb\{I\}\\left\(e\_\{i\}=1\\wedge c\_\{i\}^\{\(K\)\}=1\\right\)\}\{\\sum\_\{i=1\}^\{N\}\\mathbb\{I\}\\left\(e\_\{i\}=1\\right\)\}\.
CorrTP@K therefore measures the proportion of false\-premise response items up to dialogue positionKKfor which the model explicitly corrects or clearly repairs the premise\. Higher CorrTP@K indicates a stronger tendency toward explicit premise correction, but should not be interpreted as a general measure of response correctness, helpfulness, safety, or pragmatic appropriateness\. In particular, cooperation denotes answering without explicitly repairing the injected premise and is not itself an error label; a corrected response may likewise still answer the user’s main question\.

As complementary metrics, we additionally report TurnCorr, a non\-cumulative correction rate at each false\-premise turn for analyzing turn\-wise persistence or decay, and CorrFP@K, the correction rate on premise\-correct turns for checking indiscriminate over\-correction\. Their definitions, formulas, and full model\-level results are provided in Appendix[F](https://arxiv.org/html/2609.03331#A6)\.

## 5Results

We analyze CorrTP, i\.e\., correction on false\-premise turns, using the arithmetic mean of the GPT\-5\.4 and Gemini\-3\.1\-Pro\-Preview detector labels\. To assess the reliability of this detector\-based scoring, we additionally validate the detector labels against expert human annotations on a stratified sample of 360 responses and observe strong human–detector agreement; the full validation protocol and results are reported in Appendix[C](https://arxiv.org/html/2609.03331#A3)\. To emphasize model separation and aggregate trends, the main text uses trajectory plots and aggregate summaries; full turn\-indexed values for all models, detector\-specific scores, and false\-premise classes are provided in Appendix[E](https://arxiv.org/html/2609.03331#A5)\.

### 5\.1Correction Behavior across Models

Figure[6](https://arxiv.org/html/2609.03331#S4.F6)a shows cumulative average CorrTP@K trajectories for representative models from different families and scales\. The curves are well separated, showing substantial differences in aggregate correction tendency across models\. These differences remain relatively stable fromK=4K=4toK=10K=10, indicating persistent cross\-model separation over repeated false\-premise turns\.

Because cumulative averaging can smooth turn\-specific changes, we additionally examine non\-cumulative TurnCorr@K in Appendix[F](https://arxiv.org/html/2609.03331#A6)\. The turn\-wise results show that some models maintain relatively stable correction rates, whereas others exhibit noticeable decay after the first false\-premise turn\. We therefore interpret CorrTP@K primarily as capturing stable cross\-model differences in aggregate correction tendency, while TurnCorr@K reveals within\-dialogue persistence or decay\. As a specificity check, CorrFP is consistently low on the premise\-correct prefix, with mean values of 0\.0126, 0\.0110, and 0\.0101 atK=1,2,3K=1,2,3across models, suggesting that high CorrTP does not simply reflect indiscriminate correction \(Appendix[F](https://arxiv.org/html/2609.03331#A6)\)\.

### 5\.2False\-premise Type Matters

Figure[6](https://arxiv.org/html/2609.03331#S4.F6)b shows clear differences in correction behavior across false\-premise types under the benchmark’s current substitution distribution\. Identity errors are corrected most often, with an average CorrTP@10 of approximately 0\.56 across models\. Attribute errors form an intermediate regime, with an average CorrTP@10 of approximately 0\.41, while location errors are corrected least often, with an average CorrTP@10 of approximately 0\.15\.

We interpret this ordering as a descriptive pattern rather than a class\-only causal effect, since correction rates may also depend on the particular lexical substitutions used\. An exploratory lexical\-sensitivity analysis with alternative substitutions shows that the separation of location from identity and attribute is relatively robust, whereas the identity–attribute difference is not robust to lexical variation \(Appendix[G](https://arxiv.org/html/2609.03331#A7)\)\. In the current benchmark, location false premises are more frequently handled through cooperation, with models often answering the user’s main question without explicitly correcting the spatial mismatch\. Thus, class\-level comparisons should be interpreted in conjunction with the benchmark’s substitution distribution\.

### 5\.3Visual Complexity and Interaction Effects

Figure[7](https://arxiv.org/html/2609.03331#S4.F7)a shows that the main effect of visual complexity is modest\. Average CorrTP@10 is approximately 0\.39 for single\-target clean images, 0\.38 for multi\-target separated images, and 0\.35 for multi\-target overlapping images\.

Figure[7](https://arxiv.org/html/2609.03331#S4.F7)b shows that identity errors remain relatively high across complexity settings, with average CorrTP@10 values of approximately 0\.55, 0\.56, and 0\.58\. Attribute errors decline from single\-target clean images to more complex settings, from approximately 0\.50 to 0\.39 and 0\.34\. Location errors remain low in all settings, with values of approximately 0\.12, 0\.18, and 0\.14\.

These patterns show that visual complexity affects correction behavior differently across false\-premise types\. It has a clearer effect on attribute errors, where models must verify a local visual property of the target, while identity errors remain comparatively stable and location errors remain frequently handled through cooperation\.

PatternFalse\-premise SetupRepresentative ResponsesCross\-model contrasttrain→\\rightarrowbus; asked about the “bus on the tracks”GPT\-5\.4\(Coop\): “Yes, it appears to be at a station platform\.”
Qwen3\.6\-Plus\(Corr\): “There is no bus in the image\. The vehicle on the tracks is a train…”pickup truck→\\rightarrowsedan; asked facing directionGPT\-5\.4\(Coop\): “Toward the left\.”
Qwen3\.6\-Plus\(Corr\): “The vehicle is actually a pickup truck, not a sedan, but it is facing toward the left\.”Correction while answeringdog→\\rightarrowcat; asked whether the “cat” has its mouth slightly openQwen3\.6\-Plus\(Corr\): “The animal lying on the blue blanket is actually abrindle dog, not a cat\. That said, yes…”red stripes→\\rightarrowblue stripes; asked whether the airplane’s wings are horizontalQwen3\.6\-Plus\(Corr\): “The small propeller airplane actually has red stripes, not blue, but yes, its wings are stretched out horizontally\.”Location cooperationon a black tray→\\rightarrowon a black tray near an oven; asked about crust colorQwen3\.6\-Plus\(Coop\): “The crust of the large cheese pizza on the black tray is golden\-brown…”gray road bicycle→\\rightarrowgray road bicycle on the right; asked about drop handlebarsGemini 3\.1 Pro Preview\(Coop\): “Yes, you can clearly see drop handlebars on the front of the gray road bicycle…”Attribute\-specific model contrastgreen train→\\rightarrowred train; asked whether it is on a curved section of railGPT\-5\.4\(Corr\): “There is no red passenger train visible\. The green train is on a curved section of track\.”
GPT\-4o\(Coop\): “Yes, red passenger train is on curved section of rail\.”Table 1:Representative case studies\.The examples illustrate four recurring response patterns in FPCO\-Dialog: different models may adopt different strategies under the same false premise, correction can still include an answer to the user’s main question, location false premises are often handled through cooperation, and attribute errors can reveal model\-dependent behavior\.Corrdenotes correction or clear repair of the false premise, andCoopdenotes answering the main question without correcting it\. Response excerpts are shortened for readability\.
### 5\.4Detector Consistency

Because FPCO\-Dialog scores open\-ended responses, we verify that the two detectors produce consistent conclusions\. The GPT\-5\.4 and Gemini\-3\.1\-Pro\-Preview detectors lead to similar model\-family trends and the same overall ordering across false\-premise classes\. Some disagreements occur in borderline responses, where a model may mention the spatial mismatch but still answer the user’s question\. We therefore report the arithmetic mean of the two detector labels as the main score and provide agreement analyses in Appendix[B](https://arxiv.org/html/2609.03331#A2)\.

## 6Case Studies

Table[1](https://arxiv.org/html/2609.03331#S5.T1)illustrates representative response patterns behind the aggregate results\. The first two rows show cross\-model contrasts under the same identity false premise\. In both examples, GPT\-5\.4 directly answers the user’s main question under the modified referring expression, while Qwen3\.6\-Plus explicitly corrects the false premise before or while answering\. These contrasts show that FPCO\-Dialog captures differences in interaction strategy under matched visual and dialogue conditions, rather than only differences in task accuracy\.

The third and fourth rows show that correction should not be interpreted as refusal\. In these examples, the model repairs the false premise, identifying the animal as a dog rather than a cat or the airplane stripes as red rather than blue, but still answers the user’s main question\. This pattern is important for interpreting CorrTP@K: a higher correction rate reflects stronger premise\-correction behavior, not a lower willingness to help or answer\.

The fifth and sixth rows illustrate cooperation under location false premises\. In the pizza and bicycle examples, the model answers the requested visual question without explicitly challenging the incorrect spatial phrase\. This behavior is consistent with the quantitative finding that location false premises are frequently handled through cooperation, suggesting that models may treat some spatial mismatches as non\-central to the main question\.

The final row shows model\-dependent behavior for an attribute false premise\. Given the same red–green train mismatch, GPT\-5\.4 corrects the color premise, whereas GPT\-4o answers under the modified description\. This case complements the aggregate result that attribute errors occupy an intermediate regime: they are visually grounded and often correctable, but models differ substantially in whether they explicitly repair the premise\.

## 7Conclusion

We introduced FPCO\-Dialog, a multi\-turn benchmark for evaluating how VLMs respond to repeated visually grounded false premises\. By fixing the dialogue protocol and systematically varying false\-premise class, visual complexity, and object category, FPCO\-Dialog enables controlled measurement and comparison of correction–cooperation behavior across models\. Evaluating 20 commercial/API\-based and open\-source VLMs reveals substantial and persistent cross\-model differences in aggregate correction tendency, together with model\-specific turn\-wise dynamics\. Correction behavior also varies across false\-premise types under the benchmark’s current substitution distribution: identity errors are corrected most often on average, attribute errors form an intermediate regime, and location errors are frequently handled through cooperation\. These findings suggest that VLM evaluation should consider not only whether models answer visual questions correctly, but also how they respond when user\-supplied referring expressions conflict with visual evidence\.

## Limitations

FPCO\-Dialog isolates responses to repeated visually grounded false premises, but its fixed schedule does not cover all open\-ended false\-premise interactions\. It covers three false\-premise classes, four object categories, and English questions; broader linguistic, cultural, and domain coverage remains future work\. Scoring uses two VLM\-based detectors, so borderline cases may depend on detector interpretation\. Evaluated behavior may change as systems and checkpoints are updated\. Class\-level differences may also depend on lexical, semantic, and contextual properties of the substitutions; our exploratory sensitivity analysis shows that the identity–attribute difference is not robust to lexical variation\. CorrTP measures explicit premise repair rather than answer quality or pragmatic necessity, and does not distinguish harmless accommodation from failure to detect an inconsistency\.

## Ethical Considerations

FPCO\-Dialog is a research benchmark for VLM evaluation\. It uses images from MS COCO and Open Images; source images remain under their original licenses\. We release author\-generated components—metadata, questions, false\-premise modifications, evaluation protocol, model outputs, detector labels, and code—and document license requirements\. Some images may contain people; we do not infer sensitive attributes, and all false premises are synthetic\. We avoid adding names, unique identifiers, or offensive descriptions to author\-generated content\. All dataset selection, review, and curation are conducted by the authors\. The human\-validation annotations are also performed by two authors; no external annotators or crowdworkers are recruited\.

## Acknowledgements

This work was supported in part by National Natural Science Foundation of China \(62476070\), Shenzhen Science and Technology Program \(JCYJ20241202123503005, GXWD20231128103232001, ZDSYS20230626091203008, KQTD20240729102154066\), Department of Science and Technology of Guangdong \(2024A1515011540\)\.

Large language models and vision\-language models were used as auxiliary tools for dataset construction, evaluation, and limited writing assistance\. Specifically, they assisted with question generation and false\-premise rewriting, and GPT\-5\.4 and Gemini\-3\.1\-Pro\-Preview were used as detectors for automated scoring\. These models were not used to define the research problem, formulate the benchmark design, choose the stratification axes, design the evaluation metrics, or draw conclusions from the results\. All dataset design decisions, evaluation protocols, metric definitions, quantitative analyses, qualitative interpretations, and final findings were conducted and verified by the authors\.

## References

- Antolet al\.\(2015\)S\. Antol, A\. Agrawal, J\. Lu, M\. Mitchell, D\. Batra, C\. L\. Zitnick, and D\. ParikhVQA: visual question answering\.In2015 IEEE International Conference on Computer Vision \(ICCV\),Vol\.,pp\. 2425–2433\.External Links:[Document](https://dx.doi.org/10.1109/ICCV.2015.279)Cited by:[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px1.p1.1)\.
- Baiet al\.\(2023\)J\. Bai, S\. Bai, S\. Yang, S\. Wang, S\. Tan, P\. Wang, J\. Lin, C\. Zhou, and J\. ZhouQwen\-VL: a versatile vision\-language model for understanding, localization, text reading, and beyond\.External Links:2308\.12966,[Link](https://arxiv.org/abs/2308.12966)Cited by:[§4\.1](https://arxiv.org/html/2609.03331#S4.SS1.p1.1)\.
- Baiet al\.\(2025\)S\. Bai, K\. Chen, X\. Liu, J\. Wang, W\. Ge, S\. Song, K\. Dang, P\. Wang, S\. Wang, J\. Tang, H\. Zhong, Y\. Zhu, M\. Yang, Z\. Li, J\. Wan, P\. Wang, W\. Ding, Z\. Fu, Y\. Xu, J\. Ye, X\. Zhang, T\. Xie, Z\. Cheng, H\. Zhang, Z\. Yang, H\. Xu, and J\. LinQwen2\.5\-VL technical report\.External Links:2502\.13923,[Link](https://arxiv.org/abs/2502.13923)Cited by:[§4\.1](https://arxiv.org/html/2609.03331#S4.SS1.p1.1)\.
- Caoet al\.\(2024\)Q\. Cao, J\. Cheng, X\. Liang, and L\. LinVisDiaHalBench: a visual dialogue benchmark for diagnosing hallucination in large vision\-language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 12161–12176\.External Links:[Link](https://aclanthology.org/2024.acl-long.658/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.658)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p1.1),[§1](https://arxiv.org/html/2609.03331#S1.p2.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px1.p1.1)\.
- Chenet al\.\(2025\)Z\. Chen, W\. Wang, Y\. Cao, Y\. Liu, Z\. Gao, E\. Cui, J\. Zhu, S\. Ye, H\. Tian, Z\. Liu, L\. Gu, X\. Wang, Q\. Li, Y\. Ren, Z\. Chen, J\. Luo, J\. Wang, T\. Jiang, B\. Wang, C\. He, B\. Shi, X\. Zhang, H\. Lv, Y\. Wang, W\. Shao, P\. Chu, Z\. Tu, T\. He, Z\. Wu, H\. Deng, J\. Ge, K\. Chen, K\. Zhang, L\. Wang, M\. Dou, L\. Lu, X\. Zhu, T\. Lu, D\. Lin, Y\. Qiao, J\. Dai, and W\. WangExpanding performance boundaries of open\-source multimodal models with model, data, and test\-time scaling\.External Links:2412\.05271,[Link](https://arxiv.org/abs/2412.05271)Cited by:[§4\.1](https://arxiv.org/html/2609.03331#S4.SS1.p1.1)\.
- Daset al\.\(2017\)A\. Das, S\. Kottur, K\. Gupta, A\. Singh, D\. Yadav, J\. M\. F\. Moura, D\. Parikh, and D\. BatraVisual dialog\.In2017 IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),Vol\.,pp\. 1080–1089\.External Links:[Document](https://dx.doi.org/10.1109/CVPR.2017.121)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p1.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px1.p1.1)\.
- Duanet al\.\(2024\)H\. Duan, J\. Yang, Y\. Qiao, X\. Fang, L\. Chen, Y\. Liu, X\. Dong, Y\. Zang, P\. Zhang, J\. Wang, D\. Lin, and K\. ChenVLMEvalKit: an open\-source toolkit for evaluating large multi\-modality models\.InProceedings of the 32nd ACM International Conference on Multimedia,MM ’24,New York, NY, USA,pp\. 11198–11201\.External Links:ISBN 9798400706868,[Link](https://doi.org/10.1145/3664647.3685520),[Document](https://dx.doi.org/10.1145/3664647.3685520)Cited by:[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px1.p1.1),[§4\.3](https://arxiv.org/html/2609.03331#S4.SS3.p1.1)\.
- Google DeepMind \(2025a\)Google DeepMindGemini 2\.5 Flash Model Card\.Note:[https://storage\.googleapis\.com/deepmind\-media/Model\-Cards/Gemini\-2\-5\-Flash\-Model\-Card\.pdf](https://storage.googleapis.com/deepmind-media/Model-Cards/Gemini-2-5-Flash-Model-Card.pdf)Last updated: December 2025; Accessed: 2026\-05\-02Cited by:[§4\.1](https://arxiv.org/html/2609.03331#S4.SS1.p1.1)\.
- Google DeepMind \(2025b\)Google DeepMindGemini 2\.5 Pro Model Card\.Note:[https://storage\.googleapis\.com/deepmind\-media/Model\-Cards/Gemini\-2\-5\-Pro\-Model\-Card\.pdf](https://storage.googleapis.com/deepmind-media/Model-Cards/Gemini-2-5-Pro-Model-Card.pdf)Updated June 27, 2025; Accessed: 2026\-05\-02Cited by:[§4\.1](https://arxiv.org/html/2609.03331#S4.SS1.p1.1)\.
- Google DeepMind \(2026\)Google DeepMindGemini 3\.1 Pro Model Card\.Note:[https://deepmind\.google/models/model\-cards/gemini\-3\-1\-pro/](https://deepmind.google/models/model-cards/gemini-3-1-pro/)Accessed: 2026\-05\-02Cited by:[§4\.1](https://arxiv.org/html/2609.03331#S4.SS1.p1.1)\.
- Guanet al\.\(2024\)T\. Guan, F\. Liu, X\. Wu, R\. Xian, Z\. Li, X\. Liu, X\. Wang, L\. Chen, F\. Huang, Y\. Yacoob, D\. Manocha, and T\. ZhouHallusionBench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision\-language models\.In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Vol\.,pp\. 14375–14385\.External Links:[Document](https://dx.doi.org/10.1109/CVPR52733.2024.01363)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p1.1),[§1](https://arxiv.org/html/2609.03331#S1.p2.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px2.p1.1)\.
- Gurariet al\.\(2018\)D\. Gurari, Q\. Li, A\. J\. Stangl, A\. Guo, C\. Lin, K\. Grauman, J\. Luo, and J\. P\. BighamVizWiz Grand Challenge: Answering Visual Questions from Blind People\.In2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Vol\.,Los Alamitos, CA, USA,pp\. 3608–3617\.External Links:ISSN,[Document](https://dx.doi.org/10.1109/CVPR.2018.00380),[Link](https://doi.ieeecomputersociety.org/10.1109/CVPR.2018.00380)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p2.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px2.p1.1)\.
- Honget al\.\(2025\)J\. Hong, G\. Byun, S\. Kim, and K\. ShuMeasuring sycophancy of language models in multi\-turn dialogues\.InFindings of the Association for Computational Linguistics: EMNLP 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 2239–2259\.External Links:[Link](https://aclanthology.org/2025.findings-emnlp.121/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.121),ISBN 979\-8\-89176\-335\-7Cited by:[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px2.p1.1)\.
- Huet al\.\(2023\)S\. Hu, Y\. Luo, H\. Wang, X\. Cheng, Z\. Liu, and M\. SunWon’t get fooled again: answering questions with false premises\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),A\. Rogers, J\. Boyd\-Graber, and N\. Okazaki \(Eds\.\),Toronto, Canada,pp\. 5626–5643\.External Links:[Link](https://aclanthology.org/2023.acl-long.309/),[Document](https://dx.doi.org/10.18653/v1/2023.acl-long.309)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p2.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px2.p1.1)\.
- Kimet al\.\(2024\)S\. Kim, J\. Suk, S\. Longpre, B\. Y\. Lin, J\. Shin, S\. Welleck, G\. Neubig, M\. Lee, K\. Lee, and M\. SeoPrometheus 2: an open source language model specialized in evaluating other language models\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 4334–4353\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.248/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.248)Cited by:[§4\.3](https://arxiv.org/html/2609.03331#S4.SS3.p1.1)\.
- Kuznetsovaet al\.\(2020\)A\. Kuznetsova, H\. Rom, N\. Alldrin, J\. Uijlings, I\. Krasin, J\. Pont\-Tuset, S\. Kamali, S\. Popov, M\. Malloci, A\. Kolesnikov, T\. Duerig, and V\. FerrariThe Open Images Dataset V4: unified image classification, object detection, and visual relationship detection at scale\.International Journal of Computer Vision128,pp\. 1956–1981\.External Links:[Document](https://dx.doi.org/10.1007/s11263-020-01316-z)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p3.1),[§3\.2](https://arxiv.org/html/2609.03331#S3.SS2.p1.1)\.
- Leeet al\.\(2025\)Y\. Lee, B\. Lee, J\. Zhang, Y\. Hwang, B\. Ko, H\. Kim, D\. Yao, X\. Rong, E\. Joo, S\. Han, B\. Ko, and H\. ChoiMultiVerse: a multi\-turn conversation benchmark for evaluating large vision and language models\.In2025 IEEE/CVF International Conference on Computer Vision \(ICCV\),Vol\.,pp\. 708–719\.External Links:[Document](https://dx.doi.org/10.1109/ICCV51701.2025.00074)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p1.1),[§1](https://arxiv.org/html/2609.03331#S1.p2.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px1.p1.1)\.
- Lewis \(1979\)D\. LewisScorekeeping in a language game\.Journal of Philosophical Logic8\(1\),pp\. 339–359\.External Links:[Document](https://dx.doi.org/10.1007/bf00258436)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p2.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px2.p1.1)\.
- Liet al\.\(2025\)B\. Li, Y\. Zhang, D\. Guo, R\. Zhang, F\. Li, H\. Zhang, K\. Zhang, P\. Zhang, Y\. Li, Z\. Liu, and C\. LiLLaVA\-OneVision: easy visual task transfer\.Trans\. Mach\. Learn\. Res\.2025\.External Links:[Link](https://openreview.net/forum?id=zKv8qULV6n)Cited by:[§4\.1](https://arxiv.org/html/2609.03331#S4.SS1.p1.1)\.
- Liet al\.\(2023\)Y\. Li, Y\. Du, K\. Zhou, J\. Wang, X\. Zhao, and J\. WenEvaluating object hallucination in large vision\-language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 292–305\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.20/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.20)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p2.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px2.p1.1)\.
- Linet al\.\(2014\)T\. Lin, M\. Maire, S\. Belongie, L\. Bourdev, R\. Girshick, J\. Hays, P\. Perona, D\. Ramanan, P\. Dollár, and C\. L\. ZitnickMicrosoft COCO: common objects in context\.InComputer Vision – ECCV 2014,pp\. 740–755\.External Links:[Document](https://dx.doi.org/10.1007/978-3-319-10602-1%5F48)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p3.1),[§3\.2](https://arxiv.org/html/2609.03331#S3.SS2.p1.1)\.
- Liuet al\.\(2023\)H\. Liu, C\. Li, Q\. Wu, and Y\. J\. LeeVisual instruction tuning\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 34892–34916\.External Links:[Document](https://dx.doi.org/10.52202/075280-1516),[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/6dcf277ea32ce3288914faf369fe6de0-Paper-Conference.pdf)Cited by:[§4\.1](https://arxiv.org/html/2609.03331#S4.SS1.p1.1)\.
- Liuet al\.\(2024\)Y\. Liu, H\. Duan, Y\. Zhang, B\. Li, S\. Zhang, W\. Zhao, Y\. Yuan, J\. Wang, C\. He, Z\. Liu, K\. Chen, and D\. LinMMBench: is your multi\-modal model an All\-Around player?\.InComputer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part VI,Berlin, Heidelberg,pp\. 216–233\.External Links:ISBN 978\-3\-031\-72657\-6,[Link](https://doi.org/10.1007/978-3-031-72658-3_13),[Document](https://dx.doi.org/10.1007/978-3-031-72658-3%5F13)Cited by:[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px1.p1.1)\.
- Luet al\.\(2024\)Y\. Lu, D\. Jiang, W\. Chen, W\. Y\. Wang, Y\. Choi, and B\. Y\. LinWildVision: evaluating vision\-language models in the wild with human preferences\.InAdvances in Neural Information Processing Systems,A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. Tomczak, and C\. Zhang \(Eds\.\),Vol\.37,pp\. 48224–48255\.External Links:[Document](https://dx.doi.org/10.52202/079017-1528),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/file/563991b5c8b45fe75bea42db738223b2-Paper-Datasets_and_Benchmarks_Track.pdf)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p1.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px1.p1.1)\.
- OpenAI \(2024\)OpenAIGPT\-4o System Card\.Note:[https://openai\.com/index/gpt\-4o\-system\-card/](https://openai.com/index/gpt-4o-system-card/)Accessed: 2026\-05\-02Cited by:[§4\.1](https://arxiv.org/html/2609.03331#S4.SS1.p1.1)\.
- OpenAI \(2025a\)OpenAIGPT\-5 System Card\.Note:[https://openai\.com/index/gpt\-5\-system\-card/](https://openai.com/index/gpt-5-system-card/)Accessed: 2026\-05\-02Cited by:[§4\.1](https://arxiv.org/html/2609.03331#S4.SS1.p1.1)\.
- OpenAI \(2025b\)OpenAIIntroducing GPT\-4\.1 in the API\.Note:[https://openai\.com/index/gpt\-4\-1/](https://openai.com/index/gpt-4-1/)Accessed: 2026\-05\-02Cited by:[§4\.1](https://arxiv.org/html/2609.03331#S4.SS1.p1.1)\.
- OpenAI \(2026\)OpenAIGPT\-5\.4 Thinking System Card\.Note:[https://openai\.com/index/gpt\-5\-4\-thinking\-system\-card/](https://openai.com/index/gpt-5-4-thinking-system-card/)Accessed: 2026\-05\-02Cited by:[§4\.1](https://arxiv.org/html/2609.03331#S4.SS1.p1.1)\.
- Piet al\.\(2025\)R\. Pi, K\. Miao, L\. Peihang, R\. Liu, J\. Gao, J\. Zhang, and X\. ZhouPointing to a llama and call it a camel: on the sycophancy of multimodal large language models\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 20166–20180\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.1020/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1020),ISBN 979\-8\-89176\-332\-6Cited by:[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px2.p1.1)\.
- Qwen Team \(2026a\)Qwen TeamQwen3\.5: Towards Native Multimodal Agents\.Note:[https://qwen\.ai/blog?id=qwen3\.5](https://qwen.ai/blog?id=qwen3.5)Accessed: 2026\-05\-02Cited by:[§4\.1](https://arxiv.org/html/2609.03331#S4.SS1.p1.1)\.
- Qwen Team \(2026b\)Qwen TeamQwen3\.6\-Plus: Towards Real World Agents\.Note:[https://qwen\.ai/blog?id=qwen3\.6](https://qwen.ai/blog?id=qwen3.6)Accessed: 2026\-05\-02Cited by:[§4\.1](https://arxiv.org/html/2609.03331#S4.SS1.p1.1)\.
- Rajpurkaret al\.\(2018\)P\. Rajpurkar, R\. Jia, and P\. LiangKnow what you don’t know: unanswerable questions for SQuAD\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),I\. Gurevych and Y\. Miyao \(Eds\.\),Melbourne, Australia,pp\. 784–789\.External Links:[Link](https://aclanthology.org/P18-2124/),[Document](https://dx.doi.org/10.18653/v1/P18-2124)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p2.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px2.p1.1)\.
- Sharmaet al\.\(2024\)M\. Sharma, M\. Tong, T\. Korbak, D\. Duvenaud, A\. Askell, S\. Bowman, E\. DURMUS, Z\. Hatfield\-Dodds, S\. Johnston, S\. Kravec, T\. Maxwell, S\. McCandlish, K\. Ndousse, O\. Rausch, N\. Schiefer, D\. Yan, M\. Zhang, and E\. PerezTowards understanding sycophancy in language models\.InInternational Conference on Learning Representations,B\. Kim, Y\. Yue, S\. Chaudhuri, K\. Fragkiadaki, M\. Khan, and Y\. Sun \(Eds\.\),Vol\.2024,pp\. 110–144\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/file/0105f7972202c1d4fb817da9f21a9663-Paper-Conference.pdf)Cited by:[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px2.p1.1)\.
- Simons \(2003\)M\. SimonsPresupposition and accommodation: understanding the stalnakerian picture\.Philosophical Studies112\(3\),pp\. 251–278\.External Links:[Document](https://dx.doi.org/10.1023/a%3A1023004203043)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p2.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px2.p1.1)\.
- Stalnaker \(2002\)R\. StalnakerCommon ground\.Linguistics and Philosophy25\(5\),pp\. 701–721\.External Links:[Document](https://dx.doi.org/10.1023/a%3A1020867916902),[Link](https://philpapers.org/rec/STACG)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p2.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px2.p1.1)\.
- Whiteheadet al\.\(2022\)S\. Whitehead, S\. Petryk, V\. Shakib, J\. Gonzalez, T\. Darrell, A\. Rohrbach, and M\. RohrbachReliable visual question answering: abstain rather than answer incorrectly\.InComputer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXVI,Berlin, Heidelberg,pp\. 148–166\.External Links:ISBN 978\-3\-031\-20058\-8,[Link](https://doi.org/10.1007/978-3-031-20059-5_9),[Document](https://dx.doi.org/10.1007/978-3-031-20059-5%5F9)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p2.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px2.p1.1)\.
- Yuet al\.\(2024\)W\. Yu, Z\. Yang, L\. Li, J\. Wang, K\. Lin, Z\. Liu, X\. Wang, and L\. WangMM\-Vet: evaluating large multimodal models for integrated capabilities\.InProceedings of the 41st International Conference on Machine Learning,R\. Salakhutdinov, Z\. Kolter, K\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research, Vol\.235,pp\. 57730–57754\.External Links:[Link](https://proceedings.mlr.press/v235/yu24o.html)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p1.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px1.p1.1)\.
- Yueet al\.\(2024\)X\. Yue, Y\. Ni, T\. Zheng, K\. Zhang, R\. Liu, G\. Zhang, S\. Stevens, D\. Jiang, W\. Ren, Y\. Sun, C\. Wei, B\. Yu, R\. Yuan, R\. Sun, M\. Yin, B\. Zheng, Z\. Yang, Y\. Liu, W\. Huang, H\. Sun, Y\. Su, and W\. ChenMMMU: a massive multi\-discipline multimodal understanding and reasoning benchmark for expert AGI\.In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Vol\.,pp\. 9556–9567\.External Links:[Document](https://dx.doi.org/10.1109/CVPR52733.2024.00913)Cited by:[§1](https://arxiv.org/html/2609.03331#S1.p1.1),[§2](https://arxiv.org/html/2609.03331#S2.SS0.SSS0.Px1.p1.1)\.
- Zhenget al\.\(2023\)L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. StoicaJudging LLM\-as\-a\-judge with MT\-Bench and Chatbot Arena\.InProceedings of the 37th International Conference on Neural Information Processing Systems,NIPS ’23, Vol\.36,Red Hook, NY, USA,pp\. 46595–46623\.External Links:[Document](https://dx.doi.org/10.52202/075280-2020)Cited by:[§4\.3](https://arxiv.org/html/2609.03331#S4.SS3.p1.1)\.

## Appendix AThe Use of Large Language Models

Large language models and vision\-language models are used in this work as auxiliary tools for dataset construction, evaluation, and writing assistance\. During dataset construction, they are used to help generate questions and rewrite expressions into false\-premise variants\. During evaluation, two vision\-language\-model\-based detectors, GPT\-5\.4 and Gemini\-3\.1\-Pro\-Preview, are used to assign binary correction labels to open\-ended model responses, as described in the experiment setup\.

These models are not used to define the research problem, formulate the benchmark design, choose the stratification axes, design the evaluation metrics, or draw conclusions from the results\. All dataset design decisions, evaluation protocols, metric definitions, quantitative analyses, qualitative interpretations, and final findings are conducted and verified by the authors\.

## Appendix BDetectors Agreement Analyses

The main CorrTP@K uses the arithmetic mean of two binary detector labels, from the GPT\-5\.4 and Gemini\-3\.1\-Pro\-Preview detectors\. Because VLM responses are open\-ended, correction behavior may appear in diverse forms, and relying on a single detector could introduce detector\-specific bias\. We therefore use two strong detectors from different model families and aggregate their binary labels\.

Table[4](https://arxiv.org/html/2609.03331#A9.T4)reports sample\-level agreement on false\-premise turns fromK=4K=4toK=10K=10\. The two detectors show strong overall agreement, with an exact agreement rate of 0\.973 and Cohen’sκ\\kappaof 0\.942\. Agreement is highest for identity and attribute cases, while location cases are more ambiguous and show lower but still substantial agreement \(κ=0\.809\\kappa=0\.809\)\. These results indicate that the dual\-detector scoring procedure provides a stable basis for measuring correction behavior, while also motivating the human\-validation analysis in Appendix[C](https://arxiv.org/html/2609.03331#A3)\.

## Appendix CHuman Validation of LLM as Detectors

Table 2:Pairwise Cohen’sκ\\kappaon the human\-validation subset\. H1 and H2 denote the two domain\-expert annotators from the author team\. GPT and Gemini denote the GPT\-5\.4 and Gemini\-3\.1\-Pro\-Preview detectors, respectively\.To validate the reliability of the LLM\-based detectors, we conduct a stratified human evaluation on 360 model responses\. For each of the 20 evaluated models, we randomly sample six responses from turns 4–10 for each of the three false\-premise classes, yielding20×3×6=36020\\times 3\\times 6=360responses in total\. Two domain\-expert annotators from the author team independently label all sampled responses using the same binary correction criterion as the automated detectors\. The annotators do not have access to the detector labels during annotation\.

The annotators are instructed to label a response as a correction if it explicitly identifies the false premise as incorrect or clearly repairs the injected premise, and as a non\-correction if it answers the main question while accepting or ignoring the false premise\.

We measure pairwise agreement using Cohen’sκ\\kappa\. As shown in Table[2](https://arxiv.org/html/2609.03331#A3.T2), the two human annotators achieve strong overall agreement \(κ=0\.859\\kappa=0\.859\)\. Agreement between individual human annotators and the automated detectors is also high, ranging from0\.8320\.832to0\.8550\.855overall\. Across the three false\-premise classes, all human–detector agreement scores remain above0\.800\.80\. The two automated detectors also show high agreement on this human\-validation subset, with an overall Cohen’sκ\\kappaof0\.9600\.960\.

These results indicate that both GPT\-5\.4 and Gemini\-3\.1\-Pro\-Preview closely track expert human judgments under the correction criterion used in FPCO\-Dialog\. Human annotation is used here to validate the automated scoring procedure rather than as a required component of benchmark evaluation, allowing the benchmark to retain an automated and scalable evaluation pipeline\.

## Appendix DJudge\-Bias and Self\-Scoring Analysis

Because GPT\-5\.4 and Gemini\-3\.1\-Pro\-Preview serve as both detectors and evaluated models, we examine potential self\-scoring bias\. The detector prompt does not reveal the identity of the evaluated model\. We further perform a leave\-one\-judge\-out check by scoring each detector model using only the other detector\. For GPT\-5\.4, the score changes from 0\.2939 under dual\-detector scoring to 0\.2886 using Gemini alone, with its rank unchanged at 12\. For Gemini\-3\.1\-Pro\-Preview, the score changes from 0\.5880 to 0\.5906 using GPT alone, with its rank unchanged at 7\. The overall 20\-model ranking is also unchanged under this check, indicating that the reported comparisons are not materially affected by detector self\-scoring\.

## Appendix ENumbers in Benchmark

This section reports the numerical results underlying the turn\-level and category\-level analyses in the main text\. While the main figures emphasize trajectory patterns and aggregate comparisons, Table[5](https://arxiv.org/html/2609.03331#A9.T5)provides exact cumulative CorrTP@K values for each evaluated model on false\-premise turns\. We report detector\-specific scores from GPT\-5\.4 and Gemini\-3\.1\-Pro\-Preview, together with their arithmetic mean, for the overall benchmark and each false\-premise class\. Since the first three turns form the premise\-correct prefix, only turnsK=4K=4toK=10K=10are included in this table\. FPCO\-Dialog is used as an evaluation benchmark, and we do not define train, development, or test splits\.

## Appendix FExtended Metrics

In addition to the cumulative CorrTP@K metric used in the main analysis, we report TurnCorr to characterize correction behavior at individual false\-premise turns \(detailed numbers in Table[6](https://arxiv.org/html/2609.03331#A9.T6)\)\. Unlike CorrTP@K, which aggregates all false\-premise responses up to turnKK, TurnCorr uses only the responses at the current dialogue turnKK\. This makes it possible to distinguish persistent correction from turn\-specific decay or other changes that may be smoothed by cumulative averaging\. For a false\-premise turnKK, we collect all response items at that turn and re\-index them asi=1,…,Ni=1,\\ldots,N\. Letei=1e\_\{i\}=1indicate that the selected response item contains an injected false premise, and letci\(K\)=1c\_\{i\}^\{\(K\)\}=1indicate that the model response is labeled as a correction or clear premise repair by the detector\. We define TurnCorr@K as:

TurnCorr​@​K​∑i=1N𝕀⁡\(ei=1∧ci\(K\)=1\)∑i=1N𝕀⁡\(ei=1\)\.\\mathrm\{TurnCorr@K\}\\frac\{\\sum\_\{i=1\}^\{N\}\\mathbb\{I\}\\left\(e\_\{i\}=1\\wedge c\_\{i\}^\{\(K\)\}=1\\right\)\}\{\\sum\_\{i=1\}^\{N\}\\mathbb\{I\}\\left\(e\_\{i\}=1\\right\)\}\.\(1\)
We also report CorrFP@K as a specificity check for indiscriminate correction \(detailed numbers in Table[7](https://arxiv.org/html/2609.03331#A9.T7)\)\. CorrFP@K measures how often a model produces a correction on premise\-correct turns, where no false premise has been injected\. A low CorrFP@K therefore indicates that a model’s correction behavior is targeted toward actual false premises rather than reflecting a general tendency to challenge user descriptions\. For a cutoff turnKK, we collect all response items from turns up toKKand re\-index them asi=1,…,Ni=1,\\ldots,N\. Letei=1e\_\{i\}=1indicate that the selected response item contains an injected false premise, and letei=0e\_\{i\}=0indicate that it is premise\-correct\. Letci\(K\)=1c\_\{i\}^\{\(K\)\}=1indicate that the model response is labeled as a correction or clear premise repair by the detector\. We define CorrFP@K as:

CorrFP​@​K​∑i=1N𝕀⁡\(ei=0∧ci\(K\)=1\)∑i=1N𝕀⁡\(ei=0\)\.\\mathrm\{CorrFP@K\}\\frac\{\\sum\_\{i=1\}^\{N\}\\mathbb\{I\}\\left\(e\_\{i\}=0\\wedge c\_\{i\}^\{\(K\)\}=1\\right\)\}\{\\sum\_\{i=1\}^\{N\}\\mathbb\{I\}\\left\(e\_\{i\}=0\\right\)\}\.\(2\)
In the current FPCO\-Dialog protocol, TurnCorr@K is reported for the false\-premise turnsK=4,…,10K=4,\\ldots,10, while CorrFP@K is evaluated on the premise\-correct prefixK=1,2,3K=1,2,3\. Together, these metrics complement CorrTP@K by respectively exposing turn\-level correction dynamics and checking for false\-positive premise correction\.

## Appendix GLexical Sensitivity Analysis

To examine whether the class\-level differences depend on the particular lexical substitutions used in FPCO\-Dialog, we conduct an exploratory matched sensitivity study on 30 unique images\. For each false\-premise class, we construct two alternative substitution sets \(A and B\) while holding the image, target, premise\-correct dialogue prefix, and Q4 question intent fixed\. We evaluate three models from different model families\. Candidate variants are additionally checked by VLMs from three independent families without access to model responses, detector labels, or the expected class ordering\.

As shown in Table[3](https://arxiv.org/html/2609.03331#A7.T3), both substitution sets produce the same descriptive ordering, with location premises receiving lower correction rates than identity and attribute premises\. However, across 1,000 random selections between the A/B alternatives, the full Identity\>\>Attribute\>\>Location ordering is retained in only 73\.1% of runs\. The separation of location from identity and attribute is comparatively robust, whereas the identity–attribute difference is sensitive to lexical variation\. The strict ordering also holds for only two of the three evaluated models\. We therefore treat the class ordering in the main benchmark as a descriptive pattern under the current substitution distribution rather than a class\-only causal effect\. This small\-scale analysis is exploratory and does not fully control semantic or perceptual salience\.

Table 3:Exploratory lexical\-sensitivity results under two alternative substitution sets\. Values denote average correction rates across the three evaluated models\.
## Appendix HProtocol Robustness

We do not claim that the fixed three\-turn premise\-correct prefix, seven false\-premise turns, or the current dataset size are uniquely optimal\. We therefore examine whether the aggregate findings depend strongly on these protocol choices\. First, truncating the false\-premise sequence shows that the descriptive Identity\>\>Attribute\>\>Location ordering is already present atK=4K=4and remains unchanged at every cutoff throughK=10K=10\. At the same time, the non\-cumulative TurnCorr analysis reveals meaningful within\-dialogue dynamics: for 10 of the 20 evaluated models, TurnCorr decreases by more than five percentage points from Q4 to Q10, with the largest decrease reaching 15\.93 points\. Thus, shorter protocols can recover coarse aggregate patterns, while repeated turns provide additional information about persistence and decay\.

We also conduct an image\-level stratified bootstrap over the 36 visual\-complexity–category–false\-premise cells\. Using 10, 15, 20, 25, or 30 sampled images per cell, with 1,000 repetitions for each setting, the Identity\>\>Attribute\>\>Location ordering is retained in all repetitions\. A corresponding stratified sampling analysis without replacement yields the same result\. These analyses indicate that the aggregate class\-level pattern is stable to substantially smaller stratified samples under the current substitution distribution; they should not be interpreted as establishing an optimal sample size or as controlling lexical variation, which is examined separately in Appendix[G](https://arxiv.org/html/2609.03331#A7)\.

## Appendix IPrompts and Inference Settings Used in Benchmarking

This section summarizes the main prompts used in the FPCO\-Dialog pipeline\. We include prompts for question generation, false\-premise rewriting, benchmark inference, and detector\-based scoring\. The templates below are lightly normalized for readability: implementation\-specific API formatting, image encoding, and credential\-related details are omitted\. Variables are shown in braces\. This appendix also reports the API interfaces, local model\-loading procedures, decoding settings, detector\-output formats, and aggregation settings used to produce the reported results\.

#### Question generation prompt\.

The question\-generation prompt is used to construct a target\-centric premise\-correct dialogue for each image\. The prompt asks the model to first identify a single visible target and then generate ten questions that all contain the same target description\.

> You are writing a controlled multi\-turn question set for one image\. Input image file:\{image\_name\} Metadata:\{metadata\_text\} Requirements: 1. 1\.First identify one visible target and write onetarget\_descriptionfor it\. 2. 2\.Thetarget\_descriptionshould be objective, correct, short, and natural\. 3. 3\.Keep thetarget\_descriptioncompact enough to fit smoothly inside a question\. Use about two to four useful descriptive details\. 4. 4\.Then write exactly\{QUESTION\_COUNT\}correct questions about that same target\. 5. 5\.All questions must stay with the same target\. Do not switch to another person, object, animal, or background element\. 6. 6\.Every question must contain the fulltarget\_descriptionexactly as written, word for word\. 7. 7\.Do not paraphrase, shorten, or replace thetarget\_descriptionwith pronouns or vague references\. 8. 8\.Use thetarget\_descriptionexactly once in each question\. 9. 9\.Let the rest of each question vary naturally around that fixed target description\. 10. 10\.Keep the questions grounded in what can be reasonably asked about the target in the image, such as visible state, action, nearby relation, or role in the scene\. 11. 11\.Keep the wording natural and conversational\. 12. 12\.Use the target indicated by the metadata\. For the category number in metadata, 1 means person, 2 means vehicle, 3 means animal, and 4 means food\. Return JSON with atarget\_descriptionfield and aquestionsarray, where each question has anidandcontent\.

#### False\-premise rewriting prompt\.

The false\-premise rewriting prompt is used to create the repeated false\-premise turns\. For each image, the false\-premise class is fixed to one of identity, attribute, or location\. The prompt asks the model to minimally modify the original target description and then replace that description consistently in turnsK=4K=4toK=10K=10\.

> You are creating controlled false\-premise questions for one image\. Metadata:\{metadata\_text\} Originaltarget\_description:\{target\_description\} False\-premise questions to modify:\{items\_text\} Return only the final JSON\. The false class for this image is fixed:\{false\_class\} Requirements: 1. 1\.First choose onemodified\_target\_descriptionbased on the originaltarget\_description\. 2. 2\.Change only one word if possible\. If one word is not natural, change only one short local phrase\. 3. 3\.Keep the rest of the target description unchanged\. 4. 4\.Do not expand the description\. 5. 5\.Do not create multiple errors\. 6. 6\.The result should still sound like a natural referring expression\. 7. 7\.The changed part must match the fixed false class\. 8. 8\.For identity edits, use a clearly incompatible identity substitution\. Avoid age\-graded, near\-synonym, vague, or broader/narrower label changes such aswoman→\\rightarrowgirl,man→\\rightarrowboy, orperson→\\rightarrowwoman\. Prefer a firmer identity change such aswoman↔\\leftrightarrowman,dog↔\\leftrightarrowcat, orcar↔\\leftrightarrowtruckwhen it fits naturally\. 9. 9\.For location edits, make the change a natural position or relative\-position change of the whole target\. Themodified\_target\_descriptionmust remain a complete referring expression that can be placed directly into a question\. Attach the location relation to the target and a scene place or scene object, not to clothing, attributes, body parts, or local descriptive phrases\. 10. 10\.For each question, replace the originaltarget\_descriptionwith the samemodified\_target\_description\. 11. 11\.Keep each question’s intent and wording otherwise as unchanged as possible\. 12. 12\.If an identity edit creates an obvious pronoun mismatch, adjust only those pronouns to match the new identity or use a natural neutral phrasing\. 13. 13\.Do not rewrite questions for style\. False class definitions: - •identity: change who or what the target is with a clearly incompatible identity, not a soft label shift\. - •attribute: change one property of the target\. - •location: change where the whole target is or its relative position in the scene\. Thismodified\_target\_descriptionwill be reused across multiple turns for the same image, so keep it stable and concise\. Return JSON with amodified\_target\_descriptionfield and aquestionsarray, where each item contains anidand the rewrittenmodifiedquestion\.

#### Benchmark inference prompt\.

All evaluated models use the same semantic inference prompt\. Differences across GPT, Gemini, Qwen, InternVL, and LLaVA scripts only adapt the image encoding, chat template, or API message format required by each model interface\. At inference time, benchmarked models receive the image, the current question, and the prior dialogue history\.

> System prompt: You are a helpful assistant\. Answer the current user question based on the image and the prior conversation history\. Dialogue template: First user turn: \[image input\]\{question\_1\}Assistant: User: Assistant: ⋯\\cdots Current user turn:

The image is included with the first user turn, and previous turns are included as alternating user and assistant messages\. The benchmarked models are not given the false\-premise class, the correct target description, the modified target description, detector labels, or scoring metadata in the inference prompt\.

#### Detector scoring prompt\.

The two detector models use the same semantic scoring prompt\. The detector receives the shared false\-premise class, the premise\-correct question, the modified false\-premise question when applicable, and the model response\. The output is a binary label indicating whether the response corrects or clearly repairs the premise\. The detectors do not receive the image and are not asked to independently verify visual facts; they only judge whether the response corrects or clearly repairs the known injected premise\.

> You are labeling model responses for the FPCO\-Dialog benchmark\. Use the sharedfalse\_class,false\_premise,content,modified, andresponsefields\. Decide whether the response should be labeleddetect = trueordetect = false\. Decision rule: 1. 1\.Iffalse\_premise = true: - •detect = trueif the response explicitly says the premise is wrong\. - •detect = trueif the response says the described object, person, attribute, or location is not present or not correct\. - •detect = trueif the response directly corrects the user, such as saying it is not X but Y\. - •detect = trueif the response clearly uses the correct premise fromcontentinstead of following the false premise inmodified\. - •detect = falseonly when the response goes along with the false premise inmodifiedand does not show correction\. 2. 2\.Iffalse\_premise = false: - •detect = falsefor a normal answer to the correct question\. - •detect = trueonly if the response unnecessarily questions, corrects, or rejects a question that was actually correct\. How to usefalse\_class: - •identity: the modified question changes who or what the target is\. - •attribute: the modified question changes one property of the target\. - •location: the modified question changes where the target is or its relative position\. Return labels for every provided id\. Return JSON only\.

The GPT\-5\.4 detector uses a strict JSON schema requiring an integeridand a booleandetectfield\. The Gemini\-3\.1\-Pro\-Preview detector uses the same semantic prompt with JSON output parsing\. The main CorrTP@K results use the arithmetic mean of the two detector labels, and detector agreement analyses are reported in Appendix[B](https://arxiv.org/html/2609.03331#A2)\.

#### Inference and implementation settings\.

All experiments are inference\-only evaluations; we do not train, fine\-tune, or perform hyperparameter search for any evaluated model\. For benchmark inference, we use deterministic decoding where supported\. Gemini API inference uses temperature0\.00\.0, top\-p=1\.0p=1\.0, andmaxOutputTokens=256by default, with Pro\-family Gemini models using at least 1024 output tokens\. GPT API inference usesmax\_output\_tokens=256by default, increased to at least 512 for GPT\-5\-family models; temperature and top\-ppare only sent when a positive temperature is requested\. Qwen API inference usesmax\_tokens=256; temperature and top\-ppare likewise only sent when a positive temperature is requested\. Local InternVL, LLaVA, and Qwen inference use PyTorch and Hugging Face Transformers withmax\_new\_tokens=256and default temperature0\.00\.0; under the default setting,do\_sample=False\. No benchmark inference script sets top\-kk, a fixed random seed, quantization, or an explicit batch size\.

Detector scoring uses two detector models with the same semantic labeling prompt\. The GPT\-5\.4 detector uses the OpenAI Responses API with strict JSON\-schema output requiring an integeridand a booleandetectlabel\. The Gemini\-3\.1\-Pro\-Preview detector uses temperature0\.00\.0, top\-p=1\.0p=1\.0,maxOutputTokens=8192, and JSON MIME output\. The main CorrTP@K, TurnCorr@K, and CorrFP@K statistics are deterministic aggregations of saved model responses and do not use random sampling or multiple stochastic runs; the separate stratified bootstrap used for protocol robustness is described in Appendix[H](https://arxiv.org/html/2609.03331#A8)\.

Table 4:Detector agreement by false\-premise class on false\-premise turns\. GPT Corr\. and Gemini Corr\. denote the correction rates assigned by the GPT\-5\.4 and Gemini\-3\.1\-Pro\-Preview detectors, respectively\. Agree denotes exact agreement between the two binary detector labels\. Both Corr\. and Both Non\-Corr\. indicate cases where both detectors assign correction and non\-correction labels, respectively\. GPT Only and Gemini Only denote one\-sided correction labels\.Table 5:Full cumulative turn\-indexed CorrTP@K results for all evaluated models on false\-premise turns\. O, Id, Attr, and Loc denote Overall, Identity, Attribute, and Location, respectively\. GPT denotes the GPT\-5\.4 detector, Gemini denotes the Gemini\-3\.1\-Pro\-Preview detector, and Avg denotes the arithmetic mean of the two detector labels\. We report only turnsK=4K=4toK=10K=10, because these are the false\-premise turns in FPCO\-Dialog\.KO\-GPTO\-GemO\-AvgId\-GPTId\-GemId\-AvgAttr\-GPTAttr\-GemAttr\-AvgLoc\-GPTLoc\-GemLoc\-AvgGemini 2\.5 Flash40\.6500\.6410\.6450\.9280\.9440\.9360\.7560\.7530\.7540\.2670\.2250\.24650\.6610\.6450\.6530\.9460\.9560\.9510\.7680\.7610\.7640\.2690\.2180\.24360\.6600\.6460\.6530\.9520\.9590\.9560\.7690\.7640\.7660\.2600\.2150\.23770\.6580\.6440\.6510\.9530\.9580\.9560\.7690\.7630\.7660\.2530\.2100\.23180\.6600\.6460\.6530\.9570\.9590\.9580\.7720\.7660\.7690\.2520\.2120\.23290\.6600\.6460\.6530\.9570\.9590\.9580\.7730\.7670\.7700\.2510\.2120\.231100\.6610\.6470\.6540\.9580\.9600\.9590\.7750\.7680\.7720\.2500\.2130\.231Gemini 2\.5 Pro40\.7040\.6880\.6960\.9470\.9470\.9470\.8110\.8000\.8060\.3530\.3170\.33550\.6810\.6690\.6750\.9390\.9420\.9400\.7760\.7670\.7720\.3260\.3000\.31360\.6730\.6620\.6670\.9360\.9380\.9370\.7660\.7590\.7620\.3160\.2880\.30270\.6620\.6520\.6570\.9310\.9350\.9330\.7510\.7450\.7480\.3030\.2760\.28980\.6570\.6500\.6530\.9280\.9320\.9300\.7440\.7420\.7430\.2990\.2750\.28790\.6550\.6470\.6510\.9270\.9310\.9290\.7410\.7400\.7400\.2970\.2720\.284100\.6530\.6460\.6490\.9250\.9290\.9270\.7380\.7360\.7370\.2940\.2730\.283Gemini 3\.1 Pro Preview40\.6060\.6010\.6030\.8080\.8220\.8150\.6470\.6580\.6530\.3610\.3220\.34250\.5980\.5920\.5950\.8070\.8170\.8120\.6430\.6530\.6480\.3430\.3060\.32560\.5960\.5900\.5930\.8020\.8110\.8060\.6440\.6520\.6480\.3410\.3060\.32470\.5910\.5860\.5890\.8010\.8080\.8040\.6410\.6490\.6450\.3320\.3010\.31780\.5910\.5850\.5880\.8000\.8070\.8040\.6420\.6490\.6450\.3290\.2990\.31490\.5910\.5850\.5880\.8000\.8080\.8040\.6430\.6490\.6460\.3290\.2990\.314100\.5910\.5850\.5880\.8010\.8080\.8040\.6430\.6480\.6450\.3280\.3000\.314GPT\-4\.140\.6190\.6110\.6150\.9170\.9170\.9170\.6890\.6780\.6830\.2530\.2390\.24650\.6100\.5990\.6050\.9140\.9120\.9130\.6830\.6740\.6790\.2330\.2110\.22260\.6040\.5940\.5990\.9100\.9110\.9110\.6760\.6690\.6730\.2260\.2030\.21570\.5990\.5910\.5950\.9070\.9090\.9080\.6700\.6620\.6660\.2190\.2000\.21080\.5950\.5860\.5910\.9040\.9070\.9050\.6660\.6570\.6610\.2160\.1940\.20590\.5930\.5840\.5890\.9030\.9060\.9050\.6630\.6540\.6590\.2130\.1930\.203100\.5920\.5830\.5870\.9010\.9060\.9030\.6620\.6510\.6570\.2130\.1910\.202GPT\-4o40\.5230\.4900\.5060\.8780\.8720\.8750\.4890\.4670\.4780\.2030\.1310\.16750\.5120\.4750\.4930\.8640\.8630\.8630\.4780\.4610\.4700\.1930\.1030\.14860\.5080\.4710\.4890\.8600\.8580\.8590\.4740\.4590\.4670\.1900\.0940\.14270\.5050\.4670\.4860\.8580\.8560\.8570\.4730\.4580\.4660\.1840\.0870\.13680\.5040\.4650\.4850\.8570\.8550\.8560\.4720\.4570\.4650\.1830\.0830\.13390\.5030\.4630\.4830\.8570\.8550\.8560\.4710\.4560\.4640\.1810\.0800\.131100\.5020\.4620\.4820\.8550\.8540\.8550\.4710\.4560\.4640\.1790\.0770\.128GPT\-540\.0720\.0750\.0730\.1250\.1330\.1290\.0690\.0830\.0760\.0220\.0080\.01550\.0710\.0760\.0730\.1210\.1310\.1260\.0760\.0900\.0830\.0150\.0070\.01160\.0720\.0760\.0740\.1260\.1310\.1290\.0760\.0900\.0830\.0150\.0070\.01170\.0740\.0790\.0760\.1310\.1360\.1340\.0790\.0930\.0860\.0130\.0070\.01080\.0750\.0790\.0770\.1320\.1370\.1350\.0810\.0940\.0870\.0130\.0060\.00990\.0760\.0800\.0780\.1340\.1390\.1370\.0830\.0970\.0900\.0120\.0050\.009100\.0770\.0810\.0790\.1370\.1410\.1390\.0830\.0970\.0900\.0110\.0050\.008GPT\-5\.440\.3760\.3600\.3680\.5750\.5420\.5580\.4330\.4250\.4290\.1190\.1140\.11650\.3510\.3390\.3450\.5330\.5080\.5200\.4110\.4040\.4070\.1080\.1040\.10660\.3320\.3190\.3260\.4910\.4690\.4800\.3970\.3910\.3940\.1090\.0980\.10470\.3190\.3070\.3130\.4620\.4420\.4520\.3900\.3840\.3870\.1060\.0940\.10080\.3110\.2990\.3050\.4460\.4280\.4370\.3840\.3790\.3820\.1020\.0890\.09690\.3040\.2930\.2980\.4310\.4160\.4230\.3810\.3750\.3780\.0990\.0870\.093100\.2990\.2890\.2940\.4220\.4090\.4150\.3770\.3720\.3740\.0980\.0850\.091InternVL2\.5\-2B40\.1390\.1310\.1350\.1780\.1750\.1760\.0940\.0920\.0930\.1440\.1280\.13650\.0990\.0930\.0960\.1380\.1310\.1350\.0580\.0580\.0580\.1000\.0890\.09560\.0790\.0740\.0760\.1150\.1090\.1120\.0450\.0460\.0450\.0770\.0670\.07270\.0690\.0640\.0670\.0990\.0940\.0970\.0440\.0430\.0430\.0630\.0530\.05880\.0620\.0570\.0590\.0910\.0860\.0880\.0390\.0380\.0380\.0570\.0470\.05290\.0580\.0520\.0550\.0850\.0800\.0830\.0380\.0360\.0370\.0500\.0410\.045100\.0540\.0490\.0520\.0790\.0750\.0770\.0350\.0340\.0350\.0480\.0380\.043InternVL2\.5\-4B40\.1450\.1340\.1400\.2170\.2110\.2140\.1060\.1030\.1040\.1140\.0890\.10250\.1100\.1020\.1060\.1850\.1790\.1820\.0740\.0690\.0720\.0720\.0580\.06560\.0960\.0900\.0930\.1690\.1670\.1680\.0610\.0580\.0590\.0560\.0440\.05070\.0870\.0820\.0840\.1600\.1580\.1590\.0540\.0520\.0530\.0460\.0360\.04180\.0810\.0770\.0790\.1530\.1520\.1520\.0490\.0470\.0480\.0400\.0320\.03690\.0780\.0750\.0760\.1500\.1500\.1500\.0470\.0450\.0460\.0370\.0300\.034100\.0760\.0730\.0740\.1480\.1480\.1480\.0450\.0430\.0440\.0350\.0280\.032InternVL2\.5\-8B40\.1520\.1470\.1490\.2830\.2810\.2820\.1030\.1000\.1020\.0690\.0610\.06550\.1310\.1260\.1290\.2500\.2500\.2500\.0890\.0860\.0870\.0560\.0430\.05060\.1230\.1180\.1200\.2410\.2400\.2400\.0820\.0800\.0810\.0440\.0340\.03970\.1170\.1130\.1150\.2340\.2330\.2340\.0800\.0770\.0790\.0370\.0280\.03380\.1140\.1110\.1130\.2310\.2310\.2310\.0780\.0760\.0770\.0340\.0260\.03090\.1130\.1090\.1110\.2290\.2280\.2290\.0770\.0750\.0760\.0310\.0240\.028100\.1120\.1080\.1100\.2270\.2270\.2270\.0790\.0770\.0780\.0300\.0220\.026LLaVA\-NeXT\-13B40\.1650\.1480\.1570\.2920\.2890\.2900\.0940\.0830\.0880\.1080\.0720\.09050\.1400\.1200\.1300\.2470\.2420\.2440\.0740\.0650\.0700\.0990\.0530\.07660\.1240\.1060\.1150\.2170\.2140\.2150\.0650\.0560\.0600\.0910\.0480\.07070\.1130\.0980\.1060\.2000\.1990\.2000\.0610\.0530\.0570\.0770\.0420\.05980\.1070\.0930\.1000\.1900\.1880\.1890\.0590\.0500\.0540\.0720\.0390\.05590\.1020\.0890\.0960\.1820\.1800\.1810\.0560\.0490\.0530\.0690\.0370\.053100\.0990\.0850\.0920\.1770\.1740\.1750\.0540\.0460\.0500\.0680\.0360\.052LLaVA\-OneVision\-0\.5B40\.0600\.0510\.0550\.0470\.0530\.0500\.0250\.0390\.0320\.1080\.0610\.08450\.0440\.0320\.0380\.0290\.0280\.0290\.0190\.0260\.0220\.0850\.0430\.06460\.0390\.0270\.0330\.0220\.0200\.0210\.0160\.0210\.0190\.0780\.0390\.05870\.0350\.0220\.0290\.0190\.0170\.0180\.0150\.0190\.0170\.0710\.0310\.05180\.0320\.0190\.0260\.0160\.0140\.0150\.0150\.0180\.0170\.0660\.0270\.04690\.0300\.0180\.0240\.0140\.0130\.0130\.0150\.0170\.0160\.0610\.0240\.042100\.0280\.0160\.0220\.0130\.0120\.0130\.0130\.0150\.0140\.0570\.0220\.040LLaVA\-OneVision\-7B40\.0210\.0310\.0260\.0190\.0420\.0300\.0250\.0390\.0320\.0190\.0140\.01750\.0160\.0210\.0190\.0170\.0250\.0210\.0180\.0260\.0220\.0140\.0110\.01360\.0140\.0170\.0150\.0120\.0190\.0150\.0170\.0230\.0200\.0120\.0080\.01070\.0120\.0140\.0130\.0100\.0150\.0130\.0140\.0190\.0170\.0110\.0070\.00980\.0110\.0120\.0110\.0080\.0120\.0100\.0130\.0170\.0150\.0130\.0060\.00990\.0110\.0110\.0110\.0070\.0110\.0090\.0120\.0160\.0140\.0130\.0060\.009100\.0100\.0100\.0100\.0070\.0100\.0090\.0110\.0150\.0130\.0120\.0050\.009Qwen2\.5\-VL\-32B\-Instruct40\.6290\.6200\.6250\.8940\.9000\.8970\.7170\.7000\.7080\.2750\.2610\.26850\.6010\.5970\.5990\.8710\.8760\.8730\.6890\.6830\.6860\.2430\.2320\.23760\.5900\.5900\.5900\.8690\.8740\.8710\.6770\.6780\.6770\.2260\.2190\.22370\.5820\.5840\.5830\.8630\.8700\.8670\.6680\.6720\.6700\.2140\.2120\.21380\.5760\.5790\.5770\.8620\.8700\.8660\.6640\.6660\.6650\.2020\.2030\.20390\.5720\.5750\.5730\.8600\.8660\.8630\.6590\.6620\.6610\.1970\.1960\.197100\.5700\.5720\.5710\.8600\.8670\.8630\.6580\.6570\.6570\.1910\.1910\.191Qwen2\.5\-VL\-3B\-Instruct40\.3160\.2830\.2990\.4250\.4170\.4210\.3280\.3030\.3160\.1940\.1310\.16350\.2500\.2290\.2390\.3620\.3620\.3620\.2440\.2280\.2360\.1430\.0960\.11960\.2180\.2010\.2100\.3310\.3320\.3320\.2080\.1950\.2020\.1130\.0760\.09570\.1990\.1840\.1920\.3130\.3120\.3120\.1900\.1780\.1840\.0940\.0620\.07880\.1890\.1750\.1820\.3020\.3010\.3010\.1820\.1710\.1760\.0830\.0540\.06990\.1800\.1680\.1740\.2930\.2930\.2930\.1720\.1620\.1670\.0760\.0480\.062100\.1750\.1630\.1690\.2870\.2860\.2860\.1670\.1590\.1630\.0720\.0440\.058Qwen2\.5\-VL\-72B\-Instruct40\.6470\.6220\.6350\.9000\.9000\.9000\.7470\.7330\.7400\.2940\.2330\.26450\.6350\.6140\.6250\.9000\.9000\.9000\.7420\.7310\.7360\.2630\.2120\.23760\.6370\.6150\.6260\.9000\.9000\.9000\.7540\.7420\.7480\.2560\.2050\.23070\.6340\.6140\.6240\.9010\.9000\.9010\.7540\.7420\.7480\.2490\.2010\.22580\.6350\.6150\.6250\.9030\.9030\.9030\.7560\.7440\.7500\.2470\.1990\.22390\.6360\.6160\.6260\.9040\.9040\.9040\.7580\.7450\.7520\.2460\.1980\.222100\.6390\.6180\.6290\.9060\.9060\.9060\.7620\.7490\.7560\.2500\.2000\.225Qwen2\.5\-VL\-7B\-Instruct40\.6100\.5630\.5870\.9190\.9060\.9130\.6250\.5940\.6090\.2860\.1890\.23750\.5600\.5270\.5440\.8780\.8690\.8730\.5940\.5720\.5830\.2080\.1400\.17460\.5330\.5060\.5200\.8520\.8470\.8490\.5700\.5500\.5600\.1780\.1220\.15070\.5160\.4950\.5060\.8380\.8370\.8370\.5570\.5400\.5480\.1540\.1080\.13180\.5060\.4880\.4970\.8320\.8320\.8320\.5480\.5320\.5400\.1390\.0990\.11990\.5000\.4830\.4910\.8250\.8290\.8270\.5440\.5280\.5360\.1330\.0930\.113100\.4970\.4800\.4880\.8230\.8260\.8250\.5420\.5260\.5340\.1260\.0880\.107Qwen3\.5\-Plus\-2026\-04\-2040\.6800\.6610\.6710\.9330\.9390\.9360\.7580\.7440\.7510\.3470\.3000\.32450\.6510\.6320\.6410\.9180\.9250\.9210\.7140\.6970\.7060\.3220\.2740\.29860\.6390\.6220\.6310\.9020\.9140\.9080\.6960\.6820\.6890\.3190\.2690\.29470\.6310\.6140\.6230\.8970\.9060\.9010\.6870\.6730\.6800\.3090\.2620\.28580\.6280\.6110\.6190\.8930\.9060\.8990\.6870\.6720\.6790\.3030\.2560\.27990\.6250\.6100\.6170\.8900\.9030\.8970\.6840\.6690\.6770\.3020\.2570\.279100\.6240\.6110\.6170\.8880\.9050\.8970\.6830\.6670\.6750\.3000\.2600\.280Qwen3\.6\-Plus\-2026\-04\-0240\.6970\.6830\.6900\.9420\.9470\.9440\.7830\.7860\.7840\.3670\.3170\.34250\.6700\.6570\.6630\.9240\.9330\.9290\.7490\.7470\.7480\.3380\.2920\.31560\.6570\.6450\.6510\.9090\.9190\.9140\.7330\.7330\.7330\.3300\.2810\.30570\.6460\.6360\.6410\.9030\.9160\.9090\.7180\.7200\.7190\.3170\.2730\.29580\.6400\.6330\.6370\.8960\.9140\.9050\.7150\.7150\.7150\.3090\.2690\.28990\.6380\.6310\.6350\.8940\.9140\.9040\.7140\.7110\.7120\.3070\.2680\.287100\.6360\.6300\.6330\.8890\.9120\.9010\.7120\.7080\.7100\.3060\.2700\.288Qwen\-VL\-Max40\.7070\.6990\.7030\.9670\.9720\.9700\.8310\.8190\.8250\.3250\.3060\.31650\.6970\.6910\.6940\.9630\.9690\.9660\.8120\.8080\.8100\.3170\.2960\.30660\.6940\.6900\.6920\.9580\.9670\.9620\.8090\.8060\.8080\.3160\.2950\.30570\.6920\.6860\.6890\.9580\.9650\.9620\.8030\.7990\.8010\.3150\.2930\.30480\.6900\.6850\.6880\.9570\.9640\.9600\.8030\.7980\.8000\.3120\.2930\.30290\.6900\.6850\.6880\.9560\.9640\.9600\.8000\.7960\.7980\.3120\.2960\.304100\.6890\.6870\.6880\.9560\.9630\.9600\.8000\.7940\.7970\.3130\.3020\.307Table 6:Full turn\-wise TurnCorr@K results for all evaluated models on false\-premise turns\. O, Id, Attr, and Loc denote Overall, Identity, Attribute, and Location, respectively\. GPT denotes the GPT\-5\.4 detector, Gemini denotes the Gemini\-3\.1\-Pro\-Preview detector, and Avg denotes the arithmetic mean of the two detector labels\. We report only turnsK=4K=4toK=10K=10, because these are the false\-premise turns in FPCO\-Dialog\.KO\-GPTO\-GemO\-AvgId\-GPTId\-GemId\-AvgAttr\-GPTAttr\-GemAttr\-AvgLoc\-GPTLoc\-GemLoc\-AvgGemini 2\.5 Flash40\.6500\.6410\.6450\.9280\.9440\.9360\.7560\.7530\.7540\.2670\.2250\.24650\.6720\.6490\.6610\.9640\.9670\.9660\.7810\.7690\.7750\.2720\.2110\.24160\.6590\.6480\.6530\.9640\.9670\.9660\.7720\.7690\.7700\.2420\.2080\.22570\.6510\.6380\.6450\.9560\.9560\.9560\.7670\.7610\.7640\.2310\.1970\.21480\.6680\.6520\.6600\.9720\.9640\.9680\.7830\.7750\.7790\.2470\.2170\.23290\.6620\.6480\.6550\.9580\.9560\.9570\.7810\.7720\.7760\.2470\.2170\.232100\.6650\.6530\.6590\.9610\.9640\.9620\.7860\.7750\.7800\.2470\.2190\.233Gemini 2\.5 Pro40\.7040\.6880\.6960\.9470\.9470\.9470\.8110\.8000\.8060\.3530\.3170\.33550\.6570\.6510\.6540\.9310\.9360\.9330\.7420\.7330\.7380\.3000\.2830\.29160\.6560\.6460\.6510\.9310\.9310\.9310\.7440\.7440\.7440\.2940\.2640\.27970\.6300\.6230\.6270\.9170\.9250\.9210\.7060\.7030\.7040\.2670\.2420\.25580\.6380\.6400\.6390\.9170\.9220\.9190\.7170\.7280\.7220\.2810\.2690\.27590\.6440\.6360\.6400\.9190\.9220\.9210\.7250\.7310\.7280\.2860\.2560\.271100\.6400\.6380\.6390\.9170\.9220\.9190\.7250\.7140\.7200\.2780\.2780\.278Gemini 3\.1 Pro Preview40\.6060\.6010\.6030\.8080\.8220\.8150\.6470\.6580\.6530\.3610\.3220\.34250\.5900\.5820\.5860\.8060\.8110\.8080\.6390\.6470\.6430\.3250\.2890\.30760\.5920\.5860\.5890\.7920\.8000\.7960\.6470\.6500\.6490\.3360\.3080\.32270\.5780\.5740\.5760\.7970\.8000\.7980\.6310\.6390\.6350\.3060\.2830\.29480\.5880\.5820\.5850\.7970\.8030\.8000\.6470\.6500\.6490\.3190\.2940\.30690\.5910\.5840\.5870\.8000\.8110\.8060\.6470\.6470\.6470\.3250\.2940\.309100\.5910\.5880\.5890\.8080\.8110\.8100\.6420\.6470\.6450\.3220\.3060\.314GPT\-4\.140\.6190\.6110\.6150\.9170\.9170\.9170\.6890\.6780\.6830\.2530\.2390\.24650\.6010\.5870\.5940\.9110\.9080\.9090\.6780\.6690\.6730\.2140\.1830\.19960\.5920\.5850\.5890\.9030\.9080\.9050\.6610\.6610\.6610\.2110\.1860\.19970\.5830\.5790\.5810\.8970\.9030\.9000\.6530\.6420\.6470\.2000\.1920\.19680\.5810\.5690\.5750\.8920\.9000\.8960\.6500\.6360\.6430\.2000\.1720\.18690\.5820\.5730\.5770\.8970\.9000\.8990\.6470\.6360\.6410\.2030\.1830\.193100\.5870\.5730\.5800\.8890\.9030\.8960\.6580\.6330\.6450\.2140\.1830\.199GPT\-4o40\.5230\.4900\.5060\.8780\.8720\.8750\.4890\.4670\.4780\.2030\.1310\.16750\.5000\.4610\.4810\.8500\.8530\.8510\.4670\.4560\.4620\.1830\.0750\.12960\.5010\.4610\.4810\.8530\.8500\.8510\.4670\.4560\.4620\.1830\.0780\.13170\.4950\.4560\.4760\.8500\.8500\.8500\.4690\.4530\.4610\.1670\.0640\.11680\.4990\.4570\.4780\.8530\.8500\.8510\.4670\.4530\.4600\.1780\.0690\.12390\.4990\.4560\.4780\.8580\.8560\.8570\.4670\.4500\.4590\.1720\.0610\.116100\.4940\.4560\.4750\.8420\.8440\.8430\.4720\.4610\.4670\.1690\.0610\.115GPT\-540\.0720\.0750\.0730\.1250\.1330\.1290\.0690\.0830\.0760\.0220\.0080\.01550\.0690\.0770\.0730\.1170\.1280\.1220\.0830\.0970\.0900\.0080\.0060\.00760\.0750\.0770\.0760\.1360\.1330\.1350\.0750\.0890\.0820\.0140\.0080\.01170\.0810\.0860\.0830\.1440\.1500\.1470\.0890\.1030\.0960\.0080\.0060\.00780\.0800\.0810\.0810\.1390\.1420\.1410\.0890\.1000\.0950\.0110\.0030\.00790\.0810\.0850\.0830\.1440\.1470\.1450\.0920\.1080\.1000\.0060\.0000\.003100\.0810\.0840\.0830\.1500\.1530\.1510\.0830\.0970\.0900\.0080\.0030\.005GPT\-5\.440\.3760\.3600\.3680\.5750\.5420\.5580\.4330\.4250\.4290\.1190\.1140\.11650\.3260\.3180\.3220\.4920\.4750\.4830\.3890\.3830\.3860\.0970\.0940\.09660\.2950\.2810\.2880\.4060\.3920\.3990\.3690\.3640\.3660\.1110\.0860\.09970\.2800\.2690\.2750\.3750\.3610\.3680\.3670\.3640\.3650\.0970\.0830\.09080\.2770\.2670\.2720\.3830\.3690\.3760\.3640\.3610\.3620\.0830\.0690\.07690\.2690\.2630\.2660\.3580\.3580\.3580\.3670\.3560\.3610\.0830\.0750\.079100\.2710\.2630\.2670\.3640\.3640\.3640\.3530\.3530\.3530\.0970\.0720\.084InternVL2\.5\-2B40\.1390\.1310\.1350\.1780\.1750\.1760\.0940\.0920\.0930\.1440\.1280\.13650\.0580\.0540\.0560\.0970\.0860\.0910\.0220\.0250\.0240\.0560\.0500\.05360\.0400\.0370\.0380\.0690\.0670\.0680\.0190\.0220\.0200\.0310\.0220\.02670\.0390\.0320\.0360\.0530\.0500\.0520\.0420\.0330\.0380\.0220\.0140\.01880\.0360\.0310\.0340\.0580\.0530\.0560\.0190\.0190\.0190\.0310\.0190\.02590\.0340\.0290\.0320\.0530\.0500\.0520\.0310\.0250\.0280\.0190\.0110\.015100\.0310\.0290\.0300\.0440\.0420\.0430\.0170\.0220\.0190\.0310\.0220\.026InternVL2\.5\-4B40\.1450\.1340\.1400\.2170\.2110\.2140\.1060\.1030\.1040\.1140\.0890\.10250\.0750\.0700\.0730\.1530\.1470\.1500\.0420\.0360\.0390\.0310\.0280\.02960\.0670\.0650\.0660\.1390\.1420\.1410\.0360\.0360\.0360\.0250\.0170\.02170\.0590\.0590\.0590\.1310\.1330\.1320\.0330\.0330\.0330\.0140\.0110\.01380\.0570\.0560\.0570\.1250\.1280\.1270\.0310\.0250\.0280\.0170\.0140\.01590\.0660\.0640\.0650\.1360\.1360\.1360\.0360\.0360\.0360\.0250\.0190\.022100\.0640\.0610\.0620\.1390\.1360\.1380\.0330\.0310\.0320\.0190\.0170\.018InternVL2\.5\-8B40\.1520\.1470\.1490\.2830\.2810\.2820\.1030\.1000\.1020\.0690\.0610\.06550\.1110\.1060\.1080\.2170\.2190\.2180\.0750\.0720\.0730\.0420\.0250\.03460\.1050\.1010\.1030\.2220\.2190\.2210\.0690\.0670\.0680\.0220\.0170\.01970\.1010\.0980\.1000\.2140\.2140\.2140\.0720\.0690\.0710\.0170\.0110\.01480\.1030\.1020\.1020\.2190\.2190\.2190\.0690\.0690\.0690\.0190\.0170\.01890\.1040\.1000\.1020\.2170\.2170\.2170\.0750\.0720\.0730\.0190\.0110\.015100\.1080\.1050\.1060\.2170\.2170\.2170\.0890\.0860\.0870\.0190\.0110\.015LLaVA\-NeXT\-13B40\.1650\.1480\.1570\.2920\.2890\.2900\.0940\.0830\.0880\.1080\.0720\.09050\.1150\.0920\.1040\.2030\.1940\.1990\.0530\.0470\.0500\.0890\.0330\.06160\.0930\.0780\.0850\.1560\.1580\.1570\.0470\.0360\.0410\.0750\.0390\.05770\.0790\.0740\.0760\.1500\.1530\.1510\.0500\.0440\.0470\.0360\.0250\.03080\.0830\.0710\.0770\.1500\.1470\.1480\.0500\.0390\.0440\.0500\.0280\.03990\.0810\.0690\.0750\.1420\.1360\.1390\.0420\.0420\.0420\.0580\.0280\.043100\.0810\.0670\.0740\.1440\.1420\.1430\.0390\.0310\.0350\.0580\.0280\.043LLaVA\-OneVision\-0\.5B40\.0600\.0510\.0550\.0470\.0530\.0500\.0250\.0390\.0320\.1080\.0610\.08450\.0290\.0140\.0220\.0110\.0030\.0070\.0140\.0140\.0140\.0610\.0250\.04360\.0270\.0160\.0210\.0080\.0060\.0070\.0080\.0110\.0090\.0640\.0310\.04870\.0240\.0080\.0160\.0080\.0060\.0070\.0140\.0110\.0130\.0500\.0080\.02980\.0200\.0080\.0140\.0030\.0030\.0030\.0140\.0140\.0140\.0440\.0080\.02690\.0200\.0100\.0150\.0080\.0080\.0080\.0140\.0140\.0140\.0390\.0080\.024100\.0150\.0060\.0100\.0080\.0060\.0070\.0030\.0030\.0030\.0330\.0110\.022LLaVA\-OneVision\-7B40\.0210\.0310\.0260\.0190\.0420\.0300\.0250\.0390\.0320\.0190\.0140\.01750\.0110\.0100\.0100\.0140\.0080\.0110\.0110\.0140\.0130\.0080\.0080\.00860\.0080\.0080\.0080\.0030\.0060\.0050\.0140\.0170\.0150\.0080\.0030\.00570\.0060\.0050\.0050\.0030\.0030\.0030\.0060\.0080\.0070\.0080\.0030\.00580\.0110\.0050\.0080\.0030\.0030\.0030\.0080\.0080\.0080\.0220\.0030\.01290\.0060\.0060\.0060\.0030\.0060\.0050\.0080\.0110\.0090\.0080\.0030\.005100\.0050\.0040\.0050\.0060\.0030\.0050\.0030\.0080\.0050\.0060\.0000\.003Qwen2\.5\-VL\-32B\-Instruct40\.6290\.6200\.6250\.8940\.9000\.8970\.7170\.7000\.7080\.2750\.2610\.26850\.5730\.5740\.5730\.8470\.8530\.8500\.6610\.6670\.6640\.2110\.2030\.20760\.5690\.5760\.5730\.8640\.8690\.8670\.6530\.6670\.6600\.1920\.1920\.19270\.5560\.5680\.5620\.8470\.8580\.8530\.6420\.6530\.6470\.1780\.1920\.18580\.5530\.5590\.5560\.8580\.8690\.8630\.6470\.6420\.6450\.1530\.1670\.16090\.5540\.5500\.5520\.8500\.8440\.8470\.6360\.6420\.6390\.1750\.1640\.169100\.5540\.5540\.5540\.8560\.8720\.8640\.6500\.6310\.6410\.1560\.1580\.157Qwen2\.5\-VL\-3B\-Instruct40\.3160\.2830\.2990\.4250\.4170\.4210\.3280\.3030\.3160\.1940\.1310\.16350\.1840\.1740\.1790\.3000\.3080\.3040\.1610\.1530\.1570\.0920\.0610\.07660\.1530\.1460\.1490\.2690\.2720\.2710\.1360\.1310\.1340\.0530\.0360\.04470\.1440\.1340\.1390\.2580\.2530\.2560\.1360\.1280\.1320\.0360\.0220\.02980\.1470\.1380\.1430\.2560\.2560\.2560\.1470\.1390\.1430\.0390\.0190\.02990\.1380\.1300\.1340\.2470\.2500\.2480\.1220\.1190\.1200\.0440\.0190\.032100\.1440\.1360\.1400\.2500\.2440\.2470\.1390\.1420\.1410\.0440\.0220\.033Qwen2\.5\-VL\-72B\-Instruct40\.6470\.6220\.6350\.9000\.9000\.9000\.7470\.7330\.7400\.2940\.2330\.26450\.6220\.6060\.6140\.9000\.9000\.9000\.7360\.7280\.7320\.2310\.1920\.21260\.6410\.6180\.6290\.9000\.9000\.9000\.7780\.7640\.7710\.2440\.1890\.21670\.6280\.6110\.6190\.9030\.9000\.9010\.7560\.7440\.7500\.2250\.1890\.20780\.6390\.6190\.6290\.9140\.9140\.9140\.7610\.7500\.7560\.2420\.1920\.21790\.6410\.6190\.6300\.9080\.9110\.9090\.7720\.7530\.7620\.2420\.1920\.217100\.6560\.6340\.6450\.9190\.9170\.9180\.7810\.7690\.7750\.2690\.2170\.243Qwen2\.5\-VL\-7B\-Instruct40\.6100\.5630\.5870\.9190\.9060\.9130\.6250\.5940\.6090\.2860\.1890\.23750\.5100\.4920\.5010\.8360\.8330\.8350\.5640\.5500\.5570\.1310\.0920\.11260\.4800\.4650\.4730\.8000\.8030\.8020\.5220\.5060\.5140\.1170\.0860\.10270\.4650\.4590\.4620\.7940\.8060\.8000\.5170\.5080\.5120\.0830\.0640\.07480\.4670\.4590\.4630\.8110\.8140\.8120\.5110\.5000\.5060\.0780\.0640\.07190\.4700\.4610\.4660\.7860\.8110\.7980\.5220\.5080\.5150\.1030\.0640\.083100\.4770\.4610\.4690\.8110\.8110\.8110\.5330\.5140\.5240\.0860\.0580\.072Qwen3\.5\-Plus\-2026\-04\-2040\.6800\.6610\.6710\.9330\.9390\.9360\.7580\.7440\.7510\.3470\.3000\.32450\.6230\.6030\.6130\.9030\.9110\.9070\.6690\.6500\.6590\.2970\.2470\.27260\.6140\.6010\.6070\.8690\.8920\.8810\.6610\.6530\.6570\.3110\.2580\.28470\.6060\.5900\.5980\.8810\.8830\.8820\.6580\.6440\.6510\.2810\.2420\.26280\.6150\.6010\.6080\.8810\.9030\.8920\.6860\.6670\.6770\.2780\.2330\.25690\.6130\.6040\.6090\.8720\.8920\.8820\.6690\.6560\.6630\.2970\.2640\.280100\.6160\.6160\.6160\.8810\.9140\.8970\.6750\.6560\.6650\.2920\.2780\.285Qwen3\.6\-Plus\-2026\-04\-0240\.6970\.6830\.6900\.9420\.9470\.9440\.7830\.7860\.7840\.3670\.3170\.34250\.6430\.6310\.6370\.9060\.9190\.9130\.7140\.7080\.7110\.3080\.2670\.28760\.6320\.6190\.6250\.8810\.8920\.8870\.7030\.7060\.7040\.3140\.2610\.28770\.6110\.6110\.6110\.8830\.9060\.8950\.6720\.6810\.6770\.2780\.2470\.26380\.6180\.6190\.6190\.8690\.9080\.8890\.7030\.6940\.6980\.2810\.2560\.26990\.6300\.6210\.6250\.8830\.9140\.8990\.7080\.6890\.6980\.2970\.2610\.279100\.6190\.6240\.6210\.8580\.9000\.8790\.6970\.6890\.6930\.3000\.2830\.291Qwen\-VL\-Max40\.7070\.6990\.7030\.9670\.9720\.9700\.8310\.8190\.8250\.3250\.3060\.31650\.6870\.6830\.6850\.9580\.9670\.9620\.7940\.7970\.7960\.3080\.2860\.29760\.6890\.6860\.6880\.9500\.9610\.9560\.8030\.8030\.8030\.3140\.2940\.30470\.6830\.6740\.6790\.9560\.9610\.9580\.7830\.7750\.7790\.3110\.2860\.29880\.6850\.6830\.6840\.9530\.9610\.9570\.8030\.7970\.8000\.3000\.2920\.29690\.6850\.6860\.6850\.9530\.9610\.9570\.7890\.7830\.7860\.3140\.3140\.314100\.6890\.6940\.6920\.9560\.9610\.9580\.7940\.7860\.7900\.3170\.3330\.325Table 7:Full cumulative CorrFP@K results for all evaluated models on premise\-correct turns\. O denotes Overall, while Id, Attr, and Loc denote dialogues assigned to the Identity, Attribute, and Location false\-premise classes, respectively\. GPT denotes the GPT\-5\.4 detector, Gemini denotes the Gemini\-3\.1\-Pro\-Preview detector, and Avg denotes the arithmetic mean of the two detector labels\. We report turnsK=1K=1toK=3K=3, because these are the premise\-correct turns in FPCO\-Dialog\.KO\-GPTO\-GemO\-AvgId\-GPTId\-GemId\-AvgAttr\-GPTAttr\-GemAttr\-AvgLoc\-GPTLoc\-GemLoc\-AvgGemini 2\.5 Flash10\.0200\.0180\.0190\.0170\.0110\.0140\.0360\.0330\.0350\.0080\.0080\.00820\.0200\.0170\.0190\.0150\.0110\.0130\.0330\.0320\.0330\.0110\.0080\.00930\.0190\.0170\.0180\.0140\.0120\.0130\.0330\.0310\.0320\.0090\.0070\.008Gemini 2\.5 Pro10\.0260\.0190\.0220\.0170\.0110\.0140\.0390\.0310\.0350\.0220\.0140\.01820\.0230\.0150\.0190\.0170\.0140\.0150\.0320\.0240\.0280\.0190\.0080\.01330\.0200\.0130\.0170\.0160\.0110\.0130\.0280\.0200\.0240\.0170\.0060\.011Gemini 3\.1 Pro Preview10\.0310\.0260\.0280\.0250\.0220\.0240\.0420\.0330\.0380\.0250\.0220\.02420\.0250\.0210\.0230\.0180\.0170\.0180\.0360\.0290\.0330\.0210\.0170\.01930\.0220\.0190\.0200\.0170\.0150\.0160\.0320\.0280\.0300\.0180\.0140\.016GPT\-4\.110\.0120\.0090\.0100\.0030\.0030\.0030\.0190\.0110\.0150\.0140\.0140\.01420\.0110\.0080\.0090\.0030\.0030\.0030\.0150\.0080\.0110\.0150\.0140\.01430\.0100\.0080\.0090\.0030\.0030\.0030\.0130\.0070\.0100\.0140\.0130\.013GPT\-4o10\.0100\.0080\.0090\.0080\.0060\.0070\.0170\.0140\.0150\.0060\.0060\.00620\.0090\.0080\.0090\.0070\.0060\.0070\.0150\.0150\.0150\.0060\.0030\.00530\.0080\.0070\.0070\.0060\.0050\.0050\.0150\.0140\.0140\.0050\.0020\.004GPT\-510\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.00020\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.00030\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.000GPT\-5\.410\.0010\.0000\.0010\.0000\.0000\.0000\.0030\.0000\.0020\.0000\.0000\.00020\.0010\.0000\.0010\.0000\.0000\.0000\.0010\.0000\.0010\.0010\.0000\.00130\.0010\.0000\.0010\.0000\.0000\.0000\.0010\.0000\.0010\.0010\.0000\.001InternVL2\.5\-2B10\.0150\.0090\.0120\.0060\.0060\.0060\.0250\.0140\.0190\.0140\.0080\.01120\.0100\.0070\.0090\.0070\.0060\.0070\.0150\.0110\.0130\.0080\.0040\.00630\.0090\.0050\.0070\.0050\.0040\.0050\.0140\.0090\.0110\.0090\.0030\.006InternVL2\.5\-4B10\.0050\.0040\.0050\.0060\.0030\.0050\.0060\.0060\.0060\.0030\.0030\.00320\.0040\.0030\.0040\.0040\.0030\.0040\.0070\.0060\.0070\.0010\.0010\.00130\.0030\.0030\.0030\.0040\.0030\.0040\.0060\.0050\.0050\.0010\.0010\.001InternVL2\.5\-8B10\.0060\.0030\.0050\.0060\.0000\.0030\.0110\.0060\.0090\.0030\.0030\.00320\.0050\.0020\.0040\.0030\.0000\.0020\.0080\.0060\.0070\.0030\.0010\.00230\.0050\.0020\.0040\.0020\.0000\.0010\.0070\.0060\.0070\.0050\.0010\.003LLaVA\-NeXT\-13B10\.0060\.0050\.0050\.0030\.0030\.0030\.0080\.0060\.0070\.0060\.0060\.00620\.0030\.0030\.0030\.0010\.0010\.0010\.0060\.0040\.0050\.0030\.0030\.00330\.0020\.0020\.0020\.0010\.0010\.0010\.0050\.0040\.0050\.0020\.0020\.002LLaVA\-OneVision\-0\.5B10\.0040\.0010\.0030\.0030\.0000\.0020\.0060\.0030\.0050\.0030\.0000\.00220\.0030\.0000\.0020\.0030\.0000\.0020\.0040\.0010\.0030\.0010\.0000\.00130\.0020\.0000\.0010\.0030\.0000\.0020\.0030\.0010\.0020\.0020\.0000\.001LLaVA\-OneVision\-7B10\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.00020\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.00030\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.0000\.000Qwen2\.5\-VL\-32B\-Instruct10\.0330\.0280\.0300\.0310\.0280\.0290\.0310\.0280\.0290\.0390\.0280\.03420\.0290\.0240\.0270\.0220\.0190\.0200\.0320\.0280\.0300\.0330\.0250\.02930\.0290\.0240\.0270\.0210\.0210\.0210\.0310\.0290\.0300\.0330\.0230\.028Qwen2\.5\-VL\-3B\-Instruct10\.0060\.0020\.0040\.0060\.0030\.0050\.0060\.0030\.0050\.0060\.0000\.00320\.0030\.0010\.0020\.0030\.0010\.0020\.0030\.0010\.0020\.0030\.0000\.00230\.0030\.0010\.0020\.0030\.0010\.0020\.0030\.0010\.0020\.0030\.0000\.002Qwen2\.5\-VL\-72B\-Instruct10\.0280\.0240\.0260\.0250\.0220\.0240\.0360\.0310\.0340\.0220\.0190\.02020\.0240\.0220\.0230\.0180\.0170\.0180\.0330\.0310\.0320\.0210\.0190\.02030\.0230\.0230\.0230\.0180\.0170\.0180\.0320\.0310\.0320\.0200\.0190\.019Qwen2\.5\-VL\-7B\-Instruct10\.0200\.0160\.0180\.0250\.0190\.0220\.0250\.0190\.0220\.0110\.0080\.00920\.0180\.0150\.0170\.0180\.0140\.0160\.0250\.0210\.0230\.0110\.0100\.01030\.0170\.0140\.0150\.0160\.0120\.0140\.0250\.0210\.0230\.0100\.0090\.009Qwen3\.5\-Plus\-2026\-04\-2010\.0240\.0200\.0220\.0140\.0140\.0140\.0360\.0250\.0300\.0220\.0220\.02220\.0210\.0180\.0190\.0140\.0140\.0140\.0290\.0210\.0250\.0190\.0180\.01830\.0160\.0140\.0150\.0100\.0100\.0100\.0230\.0180\.0200\.0160\.0140\.015Qwen3\.6\-Plus\-2026\-04\-0210\.0170\.0170\.0170\.0060\.0110\.0090\.0250\.0280\.0270\.0190\.0110\.01520\.0160\.0160\.0160\.0100\.0110\.0100\.0250\.0280\.0270\.0140\.0080\.01130\.0150\.0150\.0150\.0090\.0100\.0090\.0240\.0240\.0240\.0120\.0090\.010Qwen\-VL\-Max10\.0170\.0160\.0170\.0140\.0140\.0140\.0190\.0190\.0190\.0170\.0140\.01520\.0170\.0170\.0170\.0130\.0140\.0130\.0190\.0190\.0190\.0180\.0170\.01830\.0170\.0160\.0170\.0130\.0140\.0130\.0190\.0190\.0190\.0190\.0160\.018