事实俱全，答案仍错：视觉语言模型中的整合鸿沟

# All the Facts, Still the Wrong Answer: Integration Gaps in Vision-Language Models
Source: [https://arxiv.org/html/2610.33764](https://arxiv.org/html/2610.33764)
###### Abstract

Multimodal reasoning requires models to combine multiple pieces of evidence into decisions that satisfy all relevant constraints\. Evaluations centered on final\-answer accuracy leave unclear whether failures arise from reading individual facts or using them together\. We study this question through the multi\-evidence integration gap, linking local fact recovery to global decision validity on controlled subset\-selection and path\-finding tasks\. Our evaluation combines conditional performance given correct local answers with same\-grid local queries, text\-oracle inputs, retrieval controls, and constraint ablations\. We further examine hidden\-state probes and activation replacement across network depth, token positions, and model families\. Across the evaluated models, accurate local recovery coexists with substantial global failure\. For example, Gemma\-3\-27B recovers all evaluated isolated item facts correctly but achieves only 19\.75% global validity on eight\-item visual instances\. Probes decode information about solution membership from global\-prompt states, while corresponding clean\-state replacement recovers failed decisions over model\-dependent depth ranges\. These findings show that recovering facts and representing solution\-related information are insufficient for reliable joint decisions\. They motivate evaluations and model improvements that explicitly target how available evidence is combined, alongside advances in perception and model scale\. See our implementation details at: https://github\.com/YUYANG0915/mech\.

## 1Introduction

Understanding visual evidence requires more than recognizing its individual components\. For example, a model may accurately identify each item’s cost and compatibility relations but choose items that exceed the total budget or cannot be used together\. We call an answer*globally valid*when the selected combination satisfies all task constraints simultaneously\. Producing such an answer requires both recovering individual facts and using them jointly\. This raises a fundamental question for multimodal reasoning:*when a model can correctly recover each piece of visual evidence, how reliably can it combine those pieces into a globally valid answer?*

Existing research provide complementary perspectives on this question\. Broad multimodal benchmarks assess knowledge and reasoning across diverse domains\([Yue et al\., 2024](https://arxiv.org/html/2610.33764#bib.bib1)\), while multi\-image benchmarks examine comparison, relational understanding, and reasoning over image collections and sequences\([Wang et al\., 2025](https://arxiv.org/html/2610.33764#bib.bib2);[Liu et al\., 2024a](https://arxiv.org/html/2610.33764#bib.bib3);[Meng et al\., 2025](https://arxiv.org/html/2610.33764#bib.bib4);[Wang et al\., 2024b](https://arxiv.org/html/2610.33764#bib.bib5)\)\. Research on visual composition further shows that recognizing familiar concepts does not ensure sensitivity to their relations or combinations\([Yuksekgonul et al\., 2022](https://arxiv.org/html/2610.33764#bib.bib7);[Ma et al\., 2023](https://arxiv.org/html/2610.33764#bib.bib8)\)\. Hierarchical evaluations such as ConvBench explicitly distinguish perception from higher\-level capabilities\([Liu et al\., 2024c](https://arxiv.org/html/2610.33764#bib.bib6)\)\. Together, these studies motivate separating evidence recovery from evidence use\. Nevertheless, for joint constraint satisfaction explaining an incorrect answer requires connecting three observations: whether the constituent facts are recoverable, whether the resulting decision is globally valid, and whether interventions on internal states can change that decision\.

To fully address this, we propose*multi\-evidence integration gap*: the discrepancy between successful recovery of constituent evidence and successful formation of a globally valid decision\. Alongside unconditional performance, we evaluate global validity on instances for which the same model correctly answers every constituent local question\. This conditional evaluation identifies global failures that coexist with demonstrated local competence\. Crucially, local questions and global decisions are separate computations; successful local recovery does not ensure that all facts are correctly represented or jointly used during global reasoning\. This gap motivates a question:*what separates recoverable evidence from a globally valid decision?*

To investigate this, we construct a controlled visual selection task with specified facts and verified solutions\. Models select subsets of evidence cards subject to numerical, categorical, and compatibility constraints\. We first compare global decisions with local fact recovery, including a control that presents the complete visual grid during local questioning\. We then examine alternative explanations through matched\-retrieval tasks, constraint ablations, and text\-oracle inputs\. Comparisons across eight checkpoints spanning 3B to 32B parameters test whether the phenomenon extends across model families and scales, while a separate path\-finding task examines its scope beyond subset selection\. These findings motivate evaluating multimodal reasoning through the relationship between recoverable evidence, represented information, and globally consistent decisions\. Our contributions are threefold:

- •We develop a framework that links local evidence recovery to exactly verifiable global decisions, including conditional evaluation under matched visual presentation\.
- •We characterize the multi\-evidence integration gap across model families and scales, testing its scope through retrieval controls, constraint ablations, modality comparisons, and a second structured reasoning task\.
- •We combine representation probes with controlled activation interventions to distinguish solution\-related information from successful decisions and characterize recovery effects across network depth, token scopes, and evaluated checkpoints\.

![Refer to caption](https://arxiv.org/html/2610.33764v1/overview.png)Figure 1:Overview of our study\.\(a\) A packing example illustrates the local\-to\-global gap: every price is read correctly, yet the selected combination exceeds the budget\. \(b\) Same\-grid local queries, matched retrieval, constraint variants, and visual/text inputs examine possible explanations for the gap; a weighted\-path task tests whether it extends beyond subset selection\. \(c\) Probes assess solution membership and answer correctness in global\-prompt hidden states\.
## 2Related Work

#### Multimodal Evaluation\.

Benchmarks such as MMMU assess broad multimodal understanding and reasoning\([Yue et al\., 2024](https://arxiv.org/html/2610.33764#bib.bib1)\)\. Multi\-image evaluations extend this scope to relational understanding, comparison, and reasoning across visual inputs\([Wang et al\., 2025](https://arxiv.org/html/2610.33764#bib.bib2);[Liu et al\., 2024a](https://arxiv.org/html/2610.33764#bib.bib3);[Meng et al\., 2025](https://arxiv.org/html/2610.33764#bib.bib4)\), including image sequences\([Wang et al\., 2024b](https://arxiv.org/html/2610.33764#bib.bib5)\)\. ConvBench further separates perception from higher\-level capabilities through hierarchical evaluation\([Liu et al\., 2024c](https://arxiv.org/html/2610.33764#bib.bib6)\)\. Our work complements these approaches by examining global constraint satisfaction conditional on successful constituent fact recovery, and connecting this behavioral distinction to internal interventions\.

#### Compositional Reasoning and Binding\.

Prior studies expose weaknesses in vision\-language models’ sensitivity to compositional structure, object relations, and spatial configurations\([Yuksekgonul et al\., 2022](https://arxiv.org/html/2610.33764#bib.bib7);[Ma et al\., 2023](https://arxiv.org/html/2610.33764#bib.bib8);[Kamath et al\., 2023](https://arxiv.org/html/2610.33764#bib.bib9)\)\. These findings motivate distinguishing recognition of individual concepts from their coordinated use\. Research on contextual entity binding and propositional probes additionally investigates how language models represent associations and world states\([Feng and Steinhardt, 2024](https://arxiv.org/html/2610.33764#bib.bib14);[Feng et al\., 2025](https://arxiv.org/html/2610.33764#bib.bib15)\)\. We examine whether individually recoverable visual facts jointly constrain a valid decision, rather than assuming that local recognition establishes successful integration\.

#### Long\-Context Evidence Use\.

Long\-context evaluations identify difficulties in retrieving and combining distributed information, including sensitivity to evidence position\([Liu et al\., 2024b](https://arxiv.org/html/2610.33764#bib.bib10);[Bai et al\., 2024](https://arxiv.org/html/2610.33764#bib.bib11);[Wang et al\., 2024a](https://arxiv.org/html/2610.33764#bib.bib13)\)\. NoLiMa examines retrieval beyond literal matching\([Modarressi et al\., 2025](https://arxiv.org/html/2610.33764#bib.bib12)\), while MileBench studies multimodal long\-context capabilities\([Song et al\., 2024](https://arxiv.org/html/2610.33764#bib.bib22)\)\. These directions highlight that increasing context does not guarantee effective evidence use\. Our conditional evaluation focuses on the relationship between demonstrated local competence and joint validity; the observed gap alone does not exclude retrieval difficulties during global reasoning\.

#### Mechanistic Analysis and Interventions\.

Mechanistic studies investigate factual recall and latent reasoning in language models\([Geva et al\., 2023](https://arxiv.org/html/2610.33764#bib.bib21);[Yang et al\., 2024](https://arxiv.org/html/2610.33764#bib.bib16)\), and visual information processing in vision\-language models\([Neo et al\., 2025](https://arxiv.org/html/2610.33764#bib.bib17)\)\. Particularly relevant, delimiter\-token scaling targets cross\-image information leakage through hidden\-state interventions\([Lee et al\., 2026](https://arxiv.org/html/2610.33764#bib.bib18)\)\. We study exactly verifiable joint decisions using matched and mismatched activation donors, comparing recovery across layers and token scopes\. Following work on patching methodology and interpretability pitfalls\([Zhang and Nanda, 2024](https://arxiv.org/html/2610.33764#bib.bib19);[Makelov et al\., 2024](https://arxiv.org/html/2610.33764#bib.bib20)\), we distinguish probe decodability from causal use and interpret the repair horizon as an intervention\-dependent recovery window, not the location where an error necessarily originates\.

## 3Exploratory Study

We investigate whether accurate recovery of individual visual facts is accompanied by successful joint decision\-making\. We first establish a local–global performance gap, examine whether it persists under matched visual presentation, and test its sensitivity to input modality and model scale\. These comparisons motivate further diagnosis without attributing the observed gap to a specific internal mechanism\.

#### Dataset Construction and Evaluation Setup

We construct a synthetic selection task in which each candidate is represented by a visual card containing numerical attributes, categorical attributes, and compatibility relations\. Given a grid of cards, the model must select a subset satisfying a prescribed selection size, cost and weight budgets, a color requirement, type\-count limits, and mutual compatibility\. Ground\-truth facts and valid subsets are obtained from the structured generator and exhaustive enumeration\. An answer is considered globally valid if it satisfies all constraints\.

The study contains 1,600 instances, with 400 at each candidate countk∈\{2,4,6,8\}k\\in\\\{2,4,6,8\\\}\. Each instance has a visual\-grid version and a text\-oracle version that presents its underlying facts as text while preserving the decision requirements\. We evaluate eight checkpoints: Qwen2\.5\-VL\-3B/7B/32B, Gemma\-3\-4B/12B/27B, InternVL3\-8B, and LLaVA\-OneVision\-7B\. The full\-grid local\-recovery control covers Qwen2\.5\-VL\-3B, Gemma\-3\-4B, InternVL3\-8B, and LLaVA\-OneVision\-7B\. Generation and prompting details appear in the appendix\. Increasingkkchanges both the amount of evidence and the combinatorial decision problem; the requested subset size is two atk=2k=2and three otherwise\. These comparisons therefore do not isolate a pure context\-length effect\. Construction and scoring details are provided in Appendix[A\.1](https://arxiv.org/html/2610.33764#A1.SS1), with prompt examples in Appendix[A\.4](https://arxiv.org/html/2610.33764#A1.SS4)\.

![Refer to caption](https://arxiv.org/html/2610.33764v1/figure_exploratory_study_b.png)
Figure 2:Recoverable evidence does not guarantee globally valid decisions\.\(a\) Qwen2\.5\-VL\-3B: instance\-level local recovery under single\-card and full\-grid queries, compared with unconditional visual global validity\. \(b\) Global validity conditioned on every full\-grid local query being correct\. Cells report percentages and retained sample counts; asterisks indicaten<20n<20\. Rows correspond to Qwen2\.5\-VL\-3B, Gemma\-3\-4B, InternVL3\-8B, and LLaVA\-OneVision\-7B\. \(c\) Visual and text\-oracle validity across eight checkpoints atk=8k=8\. \(d\) Within\-family visual validity across model sizes atk=8k=8; the vertical axis spans 0–26%\. Panels \(a\), \(c\), and \(d\) use 400 instances per condition\. All rates are point estimates\.
#### Local Recovery and Global Validity

We distinguish local evidence recovery from global decision\-making\. In the single\-card condition, the model extracts attributes from one card at a time\. In the full\-grid condition, it answers the same local queries while viewing the complete grid\. Local recovery is scored at the instance level: every evaluated field on every constituent card must be correct\. Global validity instead requires the selected subset to satisfy all task constraints\. Figure[2](https://arxiv.org/html/2610.33764#S3.F2)\(a\) shows a substantial separation between these capabilities\. Qwen2\.5\-VL\-3B correctly recovers all 8,000 cards in isolation\. Full\-grid instance\-level recovery also remains high, reaching 90\.5% atk=8k=8\. Nevertheless, visual global validity falls from 99\.75% atk=2k=2to 5\.75% atk=8k=8\. Thus, the ability to answer local questions about the evidence does not guarantee a valid joint decision\. However, aggregate recovery rates alone, do not reveal whether global failures are concentrated among instances with local errors\. Appendix[B\.2](https://arxiv.org/html/2610.33764#A2.SS2)reports field\-level recovery and single\-card conditional results \(Tables[A4](https://arxiv.org/html/2610.33764#A2.T4)and[A5](https://arxiv.org/html/2610.33764#A2.T5)\)\.

#### Same\-Grid Control

To examine this possibility, we align local and global predictions by instance identity within each model and measure

Pr⁡\(global valid∣all full\-grid local facts correct\)\.\\Pr\\\!\\left\(\\text\{global valid\}\\mid\\text\{all full\-grid local facts correct\}\\right\)\.\(1\)This comparison retains only instances for which every local query is answered correctly from the same grid used for the global task\. The gap persists within these subsets \(Figure[2](https://arxiv.org/html/2610.33764#S3.F2)b\)\. Atk=8k=8, Qwen2\.5\-VL\-3B achieves only 5\.80% global validity among 362 locally correct instances; InternVL3\-8B achieves 13\.37% among 389\. These results show that isolated\-card presentation cannot fully explain the observed separation\. The retained subsets nevertheless differ substantially across models: Gemma\-3\-4B retains 142 instances, whereas LLaVA\-OneVision\-7B retains only eight, making the latter estimate particularly uncertain\. Moreover, local and global prompts elicit separate computations\. This control establishes successful local recovery under matched visual presentation, not that every fact is correctly represented throughout the global decision process\. Full\-grid attribute\-level results are provided in Appendix[B\.2](https://arxiv.org/html/2610.33764#A2.SS2), Table[A6](https://arxiv.org/html/2610.33764#A2.T6)\.

#### Modality and Scale

We next examine whether changing the evidence format or increasing model scale resolves the difficulty\. Figure[2](https://arxiv.org/html/2610.33764#S3.F2)\(c\) compares visual and text\-oracle validity atk=8k=8\. Textual presentation improves performance for most evaluated checkpoints, but substantial errors remain, and the advantage is not universal: Gemma\-3\-12B performs worse with text than with visual input\. The results suggest that visual processing contributes to some failures without providing a complete explanation of the decision gap\. Larger checkpoints also improve visual validity atk=8k=8within both model families \(Figure[2](https://arxiv.org/html/2610.33764#S3.F2)d\), but do not close the gap\. Qwen2\.5\-VL\-32B and Gemma\-3\-27B both achieve 100% single\-card fact recovery, yet their visual global validity is only 18\.50% and 19\.75%, respectively\. Their text\-oracle validity reaches 28\.25% and 33\.50%\. These comparisons show that the phenomenon extends beyond the smallest checkpoints, while leaving room for further gains from scale\. Appendix[B\.1](https://arxiv.org/html/2610.33764#A2.SS1)provides the remaining candidate counts and generation\-setting breakdowns \(Tables[A2](https://arxiv.org/html/2610.33764#A2.T2)and[A3](https://arxiv.org/html/2610.33764#A2.T3)\)\.

Together, these observations establish a persistent gap between local evidence recovery and globally valid decisions in our controlled tasks\. The gap remains on instances for which every local query is answered correctly from the full grid, and neither textual presentation nor increased model scale eliminates it\. Its source remains unresolved: global decisions may require more difficult retrieval, joint constraint satisfaction, or preservation of an appropriate decision during generation\.

## 4Behavioral Explanation

Section[3](https://arxiv.org/html/2610.33764#S3)shows that the local–global gap survives a same\-grid control and remains in larger checkpoints, yet these observations do not explain which demands of the global task contribute to failure\. We now examine evidence localization, constraint satisfaction, and task specificity\. We first ask whether models can retrieve the target items when joint selection is removed\. We then restore the selection requirement while varying the active constraints, before testing whether the behavioral separation extends to path finding\. These experiments cover Qwen2\.5\-VL\-3B/7B and Gemma\-3\-4B/12B under visual and text\-oracle inputs\.

#### Retrieving Evidence versus Selecting a Valid Combination

To distinguish evidence retrieval from joint selection, we construct a matched\-retrieval variant for each of the 1,600 selection instances\. The evidence, target item set, and required output size are preserved\. We provide attribute\-based descriptors that uniquely identify the target items and ask it to return their IDs\. Accuracy measures exact recovery of this target set\. Thus, the control retains evidence localization and the item\-ID output format while removing the need to determine which combination satisfies the constraints\. Retrieval is substantially easier than joint selection \(Figure[3](https://arxiv.org/html/2610.33764#S4.F3)a\)\. At eight candidates, visual retrieval accuracy ranges from 65\.75% to 97\.50%, whereas joint\-selection validity ranges from 5\.75% to 12\.50%\. For Qwen2\.5\-VL\-7B, the corresponding rates are 97\.50% and 12\.25%; for Gemma\-3\-12B, they are 97\.25% and 12\.50%\. The separation also occurs with text input\. Consequently, difficulty locating items and returning their IDs is insufficient to account for the full decision gap\. The retrieval construction and example prompt appear in Appendices[A\.2](https://arxiv.org/html/2610.33764#A1.SS2)and[A\.4](https://arxiv.org/html/2610.33764#A1.SS4); smaller\-candidate results appear in Appendix[B\.3](https://arxiv.org/html/2610.33764#A2.SS3), Table[A7](https://arxiv.org/html/2610.33764#A2.T7)\.

![Refer to caption](https://arxiv.org/html/2610.33764v1/figure_behavioral_explanation_b.png)

Figure 3:Behavioral diagnostics of the multi\-evidence integration gap\.Q3 and Q7 denote Qwen2\.5\-VL\-3B and Qwen2\.5\-VL\-7B; G4 and G12 denote Gemma\-3\-4B and Gemma\-3\-12B\. V and T indicate visual\-grid and text\-oracle inputs, respectively\.\(a\) Retrieval versus joint selection\.Atk=8k=8, models more reliably retrieve target items from identifying attribute descriptions than select a combination satisfying all constraints in the original task\.\(b\) Constraint ablation\.Cells report valid\-answer rates \(%\) atk=8k=8; darker green indicates higher performance\. Full retains all constraint groups, whereas Num\., Cat\., and Rel\. retain only numerical, categorical, and compatibility constraints, respectively\.\(c\) Beyond subset selection\.Visual\-input exact\-path accuracy decreases as the number of graph edges increases, although the target path remains three edges long\. Each plotted condition contains 400 instances\.
#### Decomposing the Constraint Requirements

Having established that evidence can often be located without successfully solving the selection task, we next examine its constraint requirements\. For each source instance, we generate four variants: the full constraint set, numerical budgets only, categorical requirements only, and mutual compatibility only\. All variants preserve the evidence and required subset size\. We exhaustively enumerate the valid subsets under each active constraint set and accept any satisfying subset\. The four variants yield 6,400 instances per modality and model\. Comparisons use the full\-constraint variant within this diagnostic suite, whose wording differs slightly from the original prompt\. At eight candidates, visual validity is 90\.25–95\.75% for categorical constraints, 37\.25–55\.75% for numerical constraints, and 11\.75–48\.00% for compatibility constraints, compared with 6\.00–14\.25% under the full set \(Figure[3](https://arxiv.org/html/2610.33764#S4.F3)b\)\. For example, Qwen2\.5\-VL\-7B achieves 95\.25%, 49\.50%, and 29\.75% on the three individual groups, but only 13\.25% when they are combined\. Under text\-oracle inputs, categorical constraints likewise yield the highest accuracy and the full set the lowest for all four checkpoints at this candidate count; the relative difficulty of numerical and compatibility constraints varies\. Models perform well when checking categorical constraints alone, but often fail when adding up costs or weights or checking whether selected items are compatible\. However, removing constraints also makes more answers valid, so higher accuracy may reflect an easier answer criterion instead of better reasoning \(See Appendix[A\.2](https://arxiv.org/html/2610.33764#A1.SS2), Table[A1](https://arxiv.org/html/2610.33764#A1.T1), Appendix[B\.3](https://arxiv.org/html/2610.33764#A2.SS3), Table[A8](https://arxiv.org/html/2610.33764#A2.T8)for acceptance\-set sizes and remaining candidates\.\)

Table 1:Correct edge facts, incorrect paths\.Local: all\-field accuracy over 16,000 isolated edge queries\. Visual/Text: exact\-path accuracy on 400 sixteen\-edge instances\. All values are percentages\.
#### Beyond Subset Selection

These results characterize failures within subset selection\. We next ask whether models also recover individual facts correctly but fail to combine them in a different task: finding a minimum\-cost path in a directed, weighted graph\. We generate 1,600 directed, weighted graph instances, with 400 each at 4, 8, 12, and 16 edge cards\. Local queries recover individual edge attributes; the global task asks for the minimum\-cost path between specified endpoints\. We evaluate exact agreement with the generated target path and compare visual\-grid and text\-oracle inputs\. Consequently, the target path always contains three edges, so larger graphs add distractors without requiring a longer path\. The edge\-cost and identifier construction is specified in Appendix[A\.2](https://arxiv.org/html/2610.33764#A1.SS2)\(Weighted paths\), with task prompts in Appendix[A\.4](https://arxiv.org/html/2610.33764#A1.SS4)\.

All four models recover every evaluated local field across 16,000 isolated edge cards\. Nevertheless, visual exact\-path accuracy declines from 94\.00–99\.75% at four edges to 13\.25–42\.50% at sixteen edges \(Figure[3](https://arxiv.org/html/2610.33764#S4.F3)c\)\. Table[1](https://arxiv.org/html/2610.33764#S4.T1)compares modalities at the largest graph size\. Text helps some checkpoints substantially, including Gemma\-3\-12B, but reduces performance for Qwen2\.5\-VL\-3B\. Thus, the separation extends beyond selecting item subsets, while its magnitude and modality dependence vary with the model and task\. Because local edge recovery is evaluated on isolated cards, this experiment does not independently exclude difficulties caused by presenting the full graph\. Appendix[B\.3](https://arxiv.org/html/2610.33764#A2.SS3)provides the remaining text\-path results, parsing rates, and per\-field local accuracy\. A stricter construction with randomized edge identifiers, overlapping cost ranges, and same\-grid local queries yields the same separation in the supplementary materials\.

#### Implications for Internal Analysis

These controls narrow the behavioral explanation of the gap\. Models often succeed when the target evidence is identified for them, their performance depends strongly on the active constraint requirements, and failures of global decision\-making also occur in path finding despite perfect isolated evidence recovery\. None of these comparisons establishes whether the global computation has represented a valid solution or where that computation fails\. Thus, we turn to hidden\-state probes and activation interventions to distinguish solution\-related information from successful decisions and test when state replacement can change the outcome\.

## 5Internal Representations and Causal Interventions

In this section, we separate two questions: what information can be decoded from hidden states before generation, and whether replacing internal states can recover a correct decision\.

### 5\.1What Can Be Decoded from Hidden States?

#### Two Readouts of the Same Prompt States

We extract the hidden state at the final prompt token across layers, before any answer tokens are generated\. We train linear logistic\-regression probes for two targets\. A correctness probe predicts whether the model’s subsequent answer is globally valid\. A membership probe predicts whether a specified candidate ID belongs to the unique valid subset\. We train a separate membership classifier for each candidate ID and average AUC across candidates\. Thus, the membership analysis tests information about individual solution components, not whether one decoder reconstructs a complete valid answer\. Both analyses use global\-task prompts, with visual\-grid and text\-oracle inputs evaluated separately\.

#### Separating Difficulty from Decodability

Because success rates differ substantially across candidate counts, we evaluate probes separately atk=4,6,8k=4,6,8, with 400 instances per group\. Standardization and classifier fitting use training data only\. Five outer folds estimate held\-out AUC; four inner folds select the layer without consulting the outer test fold\. Membership classifiers split instances separately for each candidate ID, rather than pooling candidates from the same instance into independent examples\. To assess sensitivity to the train–test split, we repeat nested cross\-validation with five random seeds and report the mean AUC and its standard deviation across runs\. Appendix[A\.3](https://arxiv.org/html/2610.33764#A1.SS3), Appendix[C\.3](https://arxiv.org/html/2610.33764#A3.SS3)provides probe protocol, the available per\-seed scores and selected\-layer records\.

Figure 4:Decoding solution components and predicting answer correctness are different tests\.Q3/Q7 denote Qwen2\.5\-VL\-3B/7B; G4/G12 denote Gemma\-3\-4B/12B\. \(a,b\) Correctness AUC for visual\-grid and text\-oracle prompts\. \(c,d\) Candidate\-membership macro\-AUC for the same input conditions\. Points show means over five nested\-cross\-validation seeds; error bars show the standard deviation across seeds\. Each candidate\-count group contains 400 instances\. Layers are selected within the training folds\. The dashed line marks AUC0\.50\.5\.
#### Solution Components Remain Decodable

Figure[4](https://arxiv.org/html/2610.33764#S5.F4)shows that membership is more readily decoded than final correctness under these readouts\. Atk=8k=8, visual membership macro\-AUC ranges from0\.6850\.685to0\.7290\.729, whereas visual correctness AUC ranges from0\.5210\.521to0\.6110\.611\. Text membership macro\-AUC ranges from0\.6980\.698to0\.7470\.747, compared with correctness AUC of0\.5490\.549–0\.6000\.600\. Membership is much more decodable atk=4k=4and weakens as the candidate set grows\. Larger checkpoints do not uniformly improve correctness prediction: for example, the visual correctness probe is weaker for Qwen\-7B than Qwen\-3B atk=4k=4\. In conclusion, the probes can identify which items belong to the correct solution more accurately than they can predict whether the model will answer correctly\. Candidate\-specific membership results appear in Appendix[C\.3](https://arxiv.org/html/2610.33764#A3.SS3); additional\-model results and label controls appear in Appendix[C\.2](https://arxiv.org/html/2610.33764#A3.SS2)\. Appendix[C\.1](https://arxiv.org/html/2610.33764#A3.SS1)supplies the fixed\-layer profiles\.

Table 2:Activation\-patching coverage\.Counts report evaluated clean\-correct/corrupt\-wrong pairs\. Layers gives the number of tested decoder blocks for full\-prompt replacement\. Qwen2\.5\-VL\-3B also receives image\- and text\-token interventions at eight shared blocks on the same pairs\.
#### From Readout to Intervention

A probe can exploit information that the model’s own generation process does not use successfully\. Moreover, independent membership predictions need not combine into a subset satisfying every constraint\. Given this, we do not interpret these AUCs as evidence that the model has already solved the task internally\. Instead, we ask whether states obtained from a corresponding correctly answered input can alter an incorrect decision\.

### 5\.2Can State Replacement Recover Correct Decisions?

#### Solution\-Preserving Input Pairs

We modify one non\-solution item to make it superficially attractive, with low cost and weight, high value, and the required color, while leaving it incompatible with the other items\. Exhaustive enumeration verifies that the valid solution set is unchanged\. We retain clean/corrupt pairs for which the model answers the clean input correctly and the corrupted input incorrectly\. Table[2](https://arxiv.org/html/2610.33764#S5.T2)reports the pairs actually represented in the intervention outputs\. Recovery is measured on a selected, perturbation\-sensitive subset\. Appendix[D\.3](https://arxiv.org/html/2610.33764#A4.SS3), Table[A27](https://arxiv.org/html/2610.33764#A4.T27), reports the clean/corrupt outcome counts before intervention subsampling\.

#### Matched and Mismatched State Replacement

For each recipient, a matched donor is its corresponding clean input\. A mismatched donor is a different clean instance with the same candidate count and compatible prompt length; the evaluated donors have a different target subset\. We replace decoder\-block output states at the chosen prompt positions, keeping the corrupted recipient input and decoding procedure otherwise fixed\. Generation is greedy with caching disabled: the selected prompt positions are replaced on every full\-sequence forward pass, rather than only during initial prefill\. Generated\-token positions are not directly replaced\. Full\-prompt replacement patches every prompt position; restricted interventions patch only image\-token positions or their non\-image complement, which we call text tokens\.

For a layerℓ\\elland position setSS, we summarize the donor contrast as

Δℓ,S=1N​∑i=1N\(vimatched​\(ℓ,S\)−vimismatched​\(ℓ,S\)\),\\Delta\_\{\\ell,S\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\left\(v\_\{i\}^\{\\mathrm\{matched\}\}\(\\ell,S\)\-v\_\{i\}^\{\\mathrm\{mismatched\}\}\(\\ell,S\)\\right\),\(2\)whereviv\_\{i\}indicates whether the patched answer is globally valid\. Positive values mean that corresponding clean states recover valid answers more often than unrelated clean states\. We use paired bootstrap resampling of recipient pairs for pointwise 95% intervals\. This contrast measures donor\-specific recovery under the intervention\. Appendix[A\.3](https://arxiv.org/html/2610.33764#A1.SS3)provides the uncertainty protocol\.

#### Recovery Depends on Intervention Depth

For Qwen\-3B atk=4k=4, full\-prompt matched replacement recovers 95\.96% of the selected failures at block 0, compared with 0% for mismatched replacement \(Figure[5](https://arxiv.org/html/2610.33764#S5.F5)a\)\. Recovery remains high through much of the tested depth range\. At block 24, matched recovery is 87\.88%, but it falls to 3\.03% at block 27, where mismatched recovery is 2\.02%\. Thus, the advantage of matched over mismatched replacement drops sharply between blocks 24 and 27\. Corresponding clean states can recover correct answers when introduced earlier, but provide little benefit at the later tested block\.

### 5\.3Which Token Positions Support Recovery?

#### Restricting the Replaced Positions

Since full\-prompt replacement does not distinguish contributions from different prompt positions, we then compare all\-prompt, image\-token, and text\-token replacement on the same Qwen\-3B pairs and shared tested blocks \(Figure[5](https://arxiv.org/html/2610.33764#S5.F5)b,c\)\. Atk=4k=4, image\-token replacement has a donor contrast of0\.9600\.960at block 0 and0\.3740\.374at block 24\. Text\-token replacement has no positive contrast at the earliest tested blocks, but reaches0\.3640\.364at block 24\. At that block, full\-prompt replacement yields a larger contrast of0\.8790\.879\. Atk=8k=8, image\- and text\-token contrasts at block 24 are0\.3000\.300and0\.2200\.220, compared with0\.6700\.670for full\-prompt replacement\.

Figure 5:Recovery depends on both depth and replaced token positions\.All panels use Qwen2\.5\-VL\-3B with visual\-grid inputs\. \(a\) Valid\-answer rates after full\-prompt matched or mismatched replacement atk=4k=4\. \(b,c\) Matched\-minus\-mismatched valid\-answer rates for three replacement scopes atk=4k=4andk=8k=8, restricted to their shared tested blocks\. Each scope uses the same 99 and 100 recipient pairs, respectively\. Depth is the zero\-based decoder\-block index divided by the final block index\. Shading shows pointwise paired\-bootstrap 95% intervals; intervals in \(a\) are bootstrap intervals for each recovery rate\.
#### Position Is Not Information Modality

The restricted interventions show that replacing image positions and replacing text positions have different depth profiles\. Through attention, text\-token states can also carry visual information\. The later recovery from text\-token replacement is consistent with image evidence becoming accessible at text positions\. In Qwen\-3B, image\-token replacement improves recovery at early layers, whereas text\-token replacement becomes beneficial later\. At block 24, replacing all prompt positions yields a larger matched\-donor advantage than replacing either group alone\. Appendix[D\.2](https://arxiv.org/html/2610.33764#A4.SS2)completes the scope comparison atk=6k=6and reports changed\-item avoidance\.

### 5\.4Do These Patterns Persist Across Models?

#### Architecture and Scale

We next compare full\-prompt donor contrasts across model families and within Qwen and Gemma \(Figure[6](https://arxiv.org/html/2610.33764#S5.F6)\)\. Thek=4k=4comparison has the largest intervention cohorts for the additional models\. Qwen\-7B retains a positive point estimate of0\.2160\.216at normalized depth0\.7410\.741, while the Qwen\-3B contrast is near zero by its next tested depth,0\.7710\.771\. Gemma\-4B changes from0\.5590\.559at depth0\.5760\.576to−0\.029\-0\.029at0\.6060\.606; Gemma\-12B has a contrast of0\.2340\.234at0\.5530\.553and0\.0210\.021at0\.6600\.660\. InternVL3\-8B retains a positive lower pointwise interval bound through depth0\.8520\.852\. The recovery advantage persists into later layers in InternVL than in Gemma, while increasing model size produces different changes in the Qwen and Gemma families\.

Figure 6:Full\-prompt recovery profiles vary across model families and sizes\. Both panels show matched\-minus\-mismatched valid\-answer rates atk=4k=4with visual\-grid inputs\. \(a\) Q3/Q7: Qwen2\.5\-VL\-3B/7B\. \(b\) G4/G12: Gemma\-3\-4B/12B; I8: InternVL3\-8B\. Shading shows pointwise paired\-bootstrap 95% intervals\. Normalized depth aligns relative block indices\.
#### Comparing Recovery Profiles

We then compare how the matched\-donor advantage changes across depth\. For each model, the tested layers indicate the range over which replacement can recover correct answers\. We focus the cross\-model comparison onk=4k=4, where the larger cohorts provide more stable estimates\. Atk=6,8k=6,8, smaller cohorts make the estimated recovery range more sensitive to individual examples\. Appendix[D\.1](https://arxiv.org/html/2610.33764#A4.SS1)presents thek=6,8k=6,8recovery profiles for all five models and their paired donor\-contrast intervals\.

#### From Available Information to Successful Decisions

Together, the analyses distinguish three observations: individual facts can be recovered, solution membership can be decoded from global\-prompt states, and corresponding clean\-state replacement can recover some failed decisions\. The integration gap persists even when individual facts are recoverable and solution\-related information is accessible to probes\. Probes describe information accessible to an external readout; interventions show that decisions are sensitive to particular state replacements\. Their combination sharpens the integration gap without claiming that a complete solution was already represented or that one layer universally causes the failure\.

## 6Conclusion

This study presents a multi\-evidence integration gap in vision\-language models: accurate recovery of individual facts can coexist with decisions that violate their joint constraints\. Experiments on subset selection and path finding reveal this separation, while comparisons across input modalities and model sizes show that its magnitude depends on both the task and the model\. Hidden\-state probes reveal decodable solution\-membership information, and activation interventions show that corresponding clean states can recover failed decisions across model\-dependent depths and token positions\. Extract–Solve further shows that structured evidence paired with deterministic execution can recover difficult global decisions\. Together, these findings motivate multimodal evaluations and systems that connect evidence recovery to globally consistent decision\-making\.

## AI Use Statement

AI\-assisted tools were used during manuscript preparation for language editing\. All citations were selected and checked by the authors\. The authors take full responsibility for the accuracy and integrity of the work\.

## Ethics Statement

Our experiments use synthetic visual reasoning tasks and publicly available model checkpoints; they involve no human participants or personal data\. The central risk highlighted by our results is practical: a model can read every relevant fact correctly and still reach a decision that violates their joint constraints\. In settings where several pieces of evidence must be combined, strong perception scores alone may give a misleading picture of reliability\. Such systems should be evaluated on the validity of their final decisions and paired with independent checks and human oversight when errors carry meaningful consequences\. We release our code to make the evidence behind these findings auditable and to support further work on reliable multi\-evidence reasoning\.

## Reproducibility Statement

The anonymous repo contains the dataset generators, prompts, inference pipelines, exact\-match evaluators, behavioral controls, hidden\-state probes, activation\-replacement procedures, Extract–Solve analysis, statistical summaries, and figure\-generation scripts used in this study\. The release also provides model identifiers, loader configurations, random seeds, sample counts, software requirements, portable Slurm templates, and unit tests\.

## References

- Baiet al\.\(2024\)Y\. Bai, X\. Lv, J\. Zhang, H\. Lyu, J\. Tang, Z\. Huang, Z\. Du, X\. Liu, A\. Zeng, L\. Hou,et al\.Longbench: a bilingual, multitask benchmark for long context understanding\.InProceedings of the 62nd annual meeting of the association for computational linguistics \(volume 1: Long papers\),pp\. 3119–3137\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px3.p1.1)\.
- Fenget al\.\(2025\)J\. Feng, S\. Russell, and J\. SteinhardtMonitoring latent world states in language models with propositional probes\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 19337–19359\.Cited by:[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px2.p1.1)\.
- Feng and Steinhardt \(2024\)J\. Feng and J\. SteinhardtHow do language models bind entities in context?\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 36391–36413\.Cited by:[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px2.p1.1)\.
- Gevaet al\.\(2023\)M\. Geva, J\. Bastings, K\. Filippova, and A\. GlobersonDissecting recall of factual associations in auto\-regressive language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 12216–12235\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px4.p1.1)\.
- Kamathet al\.\(2023\)A\. Kamath, J\. Hessel, and K\. ChangWhat’s “up” with vision\-language models? investigating their struggle with spatial reasoning\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 9161–9175\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px2.p1.1)\.
- Leeet al\.\(2026\)M\. Lee, Y\. Park, D\. Hwang, Y\. Kim, S\. J\. Oh, and J\. ChoeEnhancing multi\-image understanding through delimiter token scaling\.arXiv preprint arXiv:2602\.01984\.Cited by:[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px4.p1.1)\.
- Liet al\.\(2025\)T\. Li, J\. Zhang, Y\. Rao, and Y\. ChengUnveiling the compositional ability gap in vision\-language reasoning model\.arXiv preprint arXiv:2505\.19406\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2024a\)H\. Liu, X\. Zhang, H\. Xu, Y\. Shi, C\. Jiang, M\. Yan, J\. Zhang, F\. Huang, C\. Yuan, B\. Li,et al\.Mibench: evaluating multimodal large language models over multiple images\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 22417–22428\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px2.p1.1),[Appendix I](https://arxiv.org/html/2610.33764#A9.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.33764#S1.p2.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2024b\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the middle: how language models use long contexts\.Transactions of the association for computational linguistics12,pp\. 157–173\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px3.p1.1)\.
- Liuet al\.\(2024c\)S\. Liu, K\. Ying, H\. Zhang, Y\. Yang, Y\. Lin, T\. Zhang, C\. Li, Y\. Qiao, P\. Luo, W\. Shao,et al\.Convbench: a multi\-turn conversation evaluation benchmark with hierarchical capability for large vision\-language models\.arXiv preprint arXiv:2403\.20194\.Cited by:[§1](https://arxiv.org/html/2610.33764#S1.p2.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px1.p1.1)\.
- Maet al\.\(2023\)Z\. Ma, J\. Hong, M\. O\. Gul, M\. Gandhi, I\. Gao, and R\. Krishna@ CREPE: can vision\-language foundation models reason compositionally?\.In2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 10910–10921\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px1.p1.1),[Appendix I](https://arxiv.org/html/2610.33764#A9.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.33764#S1.p2.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px2.p1.1)\.
- Makelovet al\.\(2024\)A\. Makelov, G\. Lange, A\. Geiger, and N\. NandaIs this the subspace you are looking for? an interpretability illusion for subspace activation patching\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 26486–26515\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px4.p1.1)\.
- Menget al\.\(2025\)F\. Meng, J\. Wang, C\. Li, Q\. Lu, H\. Tian, T\. Yang, J\. Liao, X\. Zhu, J\. Dai, Y\. Qiao,et al\.Mmiu: multimodal multi\-image understanding for evaluating large vision\-language models\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 38405–38453\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px2.p1.1),[Appendix I](https://arxiv.org/html/2610.33764#A9.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.33764#S1.p2.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px1.p1.1)\.
- Modarressiet al\.\(2025\)A\. Modarressi, H\. Deilamsalehy, F\. Dernoncourt, T\. Bui, R\. A\. Rossi, S\. Yoon, and H\. SchützeNolima: long\-context evaluation beyond literal matching\.arXiv preprint arXiv:2502\.05167\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px3.p1.1)\.
- Neoet al\.\(2025\)C\. Neo, L\. Ong, P\. Torr, M\. Geva, D\. Krueger, and F\. BarezTowards interpreting visual information processing in vision\-language models\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 57172–57189\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px4.p1.1)\.
- Songet al\.\(2024\)D\. Song, S\. Chen, G\. H\. Chen, F\. Yu, X\. Wan, and B\. WangMilebench: benchmarking mllms in long context\.arXiv preprint arXiv:2404\.18532\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px3.p1.1)\.
- Wanget al\.\(2025\)F\. Wang, X\. Fu, J\. Y\. Huang, Z\. Li, Q\. Liu, X\. Liu, M\. D\. Ma, N\. Xu, W\. Zhou, K\. Zhang,et al\.Muirbench: a comprehensive benchmark for robust multi\-image understanding\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 62624–62650\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px2.p1.1),[Appendix I](https://arxiv.org/html/2610.33764#A9.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.33764#S1.p2.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px1.p1.1)\.
- Wanget al\.\(2024a\)M\. Wang, L\. Chen, F\. Cheng, S\. Liao, X\. Zhang, B\. Wu, H\. Yu, N\. Xu, L\. Zhang, R\. Luo,et al\.Leave no document behind: benchmarking long\-context llms with extended multi\-doc qa\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 5627–5646\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px3.p1.1)\.
- Wanget al\.\(2024b\)X\. Wang, Y\. Zhou, X\. Liu, H\. Lu, Y\. Xu, F\. He, J\. Yoon, T\. Lu, F\. Liu, G\. Bertasius,et al\.Mementos: a comprehensive benchmark for multimodal large language model reasoning over image sequences\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 416–442\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.33764#S1.p2.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px1.p1.1)\.
- Yanget al\.\(2024\)S\. Yang, E\. Gribovskaya, N\. Kassner, M\. Geva, and S\. RiedelDo large language models latently perform multi\-hop reasoning?\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 10210–10229\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px4.p1.1)\.
- Yueet al\.\(2024\)X\. Yue, Y\. Ni, K\. Zhang, T\. Zheng, R\. Liu, G\. Zhang, S\. Stevens, D\. Jiang, W\. Ren, Y\. Sun,et al\.Mmmu: a massive multi\-discipline multimodal understanding and reasoning benchmark for expert agi\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 9556–9567\.Cited by:[Appendix I](https://arxiv.org/html/2610.33764#A9.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.33764#S1.p2.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px1.p1.1)\.
- Yuksekgonulet al\.\(2022\)M\. Yuksekgonul, F\. Bianchi, P\. Kalluri, D\. Jurafsky, and J\. ZouWhen and why vision\-language models behave like bags\-of\-words, and what to do about it?\.arXiv preprint arXiv:2210\.01936\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px1.p1.1),[Appendix I](https://arxiv.org/html/2610.33764#A9.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.33764#S1.p2.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px2.p1.1)\.
- Zhang and Nanda \(2024\)F\. Zhang and N\. NandaTowards best practices of activation patching in language models: metrics and methods\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 1651–1678\.Cited by:[Appendix J](https://arxiv.org/html/2610.33764#A10.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2610.33764#S2.SS0.SSS0.Px4.p1.1)\.

## Appendix Contents

Appendix A: Experimental Construction and Evaluation\.A

A\.1 Selection Instances\.A\.1

A\.2 Diagnostic Tasks\.A\.2

A\.3 Probe and Intervention Protocols\.A\.3

A\.4 Representative Prompts\.A\.4

Appendix B: Extended Behavioral Results\.B

B\.1 Candidate Count and Model Scale\.B\.1

B\.2 Local Recovery and Conditional Decisions\.B\.2

B\.3 Retrieval, Constraint Requirements, and Paths\.B\.3

Appendix C: Extended Probing Results\.C

C\.1 Information Across Layers\.C\.1

C\.2 Additional Models and Label Controls\.C\.2

C\.3 Split Sensitivity and Candidate Membership\.C\.3

Appendix D: Extended Activation\-Patching Results\.D

D\.1 Recovery at Larger Candidate Counts\.D\.1

D\.2 Token Positions and Changed\-Item Selection\.D\.2

D\.3 Selection of Intervention Cohorts\.D\.3

Appendix E: Uncertainty in Behavioral Results\.E

Appendix F: Relation to the Main Conclusions\.F

Appendix G: Prompting Controls\.G

G\.1 Explicit Reasoning Budget\.G\.1

Appendix H: A Constructive Direction: Extract–Solve\.H

Appendix I: Discussion\.I

Appendix J: Extended Related Work\.J

## Appendix AExperimental Construction and Evaluation

This appendix supplements the behavioral and internal analyses with experimental specifications, disaggregated results, and uncertainty estimates\. The tables and figures use the completed benchmark runs and final intervention cohorts\. Candidate counts and conditions already shown in the main figures are referenced there; additional conditions and finer\-grained breakdowns are presented below\.

#### Models and notation

Q3, Q7, and Q32 denote Qwen2\.5\-VL\-3B, \-7B, and \-32B; G4, G12, and G27 denote Gemma\-3\-4B, \-12B, and \-27B; I8 denotes InternVL3\-8B; and L7 denotes LLaVA\-OneVision\-7B\. These abbreviations identify checkpoints throughout the supplementary tables\. V and T denote visual\-grid and text\-oracle global inputs\. All eight models have selection and isolated local evaluations\. The diagnostic experiments use Q3, Q7, G4, and G12\. Same\-grid local controls additionally cover I8 and L7, while full\-prompt interventions cover Q3, Q7, G4, G12, and I8\. The largest checkpoints contribute behavioral scaling evidence\.

### A\.1Selection Instances

The selection benchmark contains 1,600 instances, with 400 each at two, four, six, and eight candidate items\. Each item has cost, weight, value, color, type, and a compatibility list\. The requested subset contains two items when there are two candidates and three items otherwise\. A valid answer contains the requested number of distinct items, respects the cost and weight budgets, contains the required color, satisfies the type\-count restriction, and satisfies mutual compatibility\. The rendered value field is queried locally but is not an optimization objective in this task\.

The generator enumerates candidate subsets and retains instances with a unique valid solution\. Within each candidate count, the nominal conflict settings zero and one are crossed with distractor ratios zero and 0\.25, yielding 100 instances per cell\. Realized modifications are limited by the number of items outside the solution\. In particular, the two\-candidate condition requires selecting both candidates and serves as a simple endpoint of the benchmark\.

#### Inputs and scoring

Visual inputs render the evidence cards together in a grid\. Text\-oracle inputs provide their attributes symbolically with the selection question\. Global scoring compares the normalized predicted item set with the enumerated valid solutions\. Local questions request all six attributes of a specified item\. Field accuracy measures individual attribute recovery; all\-field accuracy requires all attributes to be correct\. Instance\-level local success requires every constituent item query to be correct\. Conditional global validity is computed on that retained instance subset, with its denominator reported explicitly\.

Isolated\-card queries and same\-grid queries measure recovery under different input conditions\. The same\-grid control preserves the local question and labels while replacing the isolated image with the full evidence grid\. This provides the closer behavioral comparison with global selection\. Answers to separate local queries establish recoverability under those queries; internal probes subsequently examine states produced by the global prompt itself\.

### A\.2Diagnostic Tasks

#### Retrieval and constraint requirements

The retrieval diagnostic contains 1,600 matched instances\. Descriptors identify the target cards uniquely, so the model can recover the requested items without solving the joint constraints\. The constraint diagnostic contains 6,400 instances: full, numerical\-only, categorical\-only, and compatibility\-only variants of each source instance\. Each variant retains the required subset size\. Scoring enumerates the solutions permitted by its active constraints\. Relaxing requirements can increase the number of acceptable answers, as quantified below; both the computational requirements and the acceptance set therefore contribute to the observed differences\.

Table A1:Mean number of valid subsets after constraint ablation\. Enumerated on the final 400 instances per candidate count\. Required subset size remains fixed; this quantifies the change in the answer acceptance criterion\.kkFullNumericalCategoricalCompatibility21\.001\.001\.001\.0041\.001\.963\.381\.1961\.005\.8414\.752\.0181\.0014\.8736\.313\.36
#### Weighted paths

The second task contains 1,600 directed weighted graph instances, with 400 each at four, eight, twelve, and sixteen edge cards\. The question asks for the minimum\-cost path from A to H among nodes A–H\. Each target path has three edges and two intermediate nodes\. Target\-edge costs range from one to four; distractor\-edge costs exceed the target path’s total cost by five to fifteen\. Graph size therefore increases the surrounding evidence while the target path length remains fixed\. Local questions recover source, destination, and cost from individual edge cards; global scoring compares the predicted ordered node sequence with the generated target path\.

Target edges receive identifiers Edge 01–03 before presentation order is shuffled\. Consequently, edge identifiers and individual edge costs provide cues to target membership\. The experiment establishes a behavioral separation within this controlled construction\. Generalization to arbitrary graph search would require additional constructions that remove these cues\. The 16,000 local edge queries use isolated cards, so the same\-grid control for selection remains the evidence addressing full\-grid local recovery\.

### A\.3Probe and Intervention Protocols

#### Layer\-wise decoding

Probes use the final prompt\-position hidden state before answer generation\. Hidden\-state index zero denotes the embedding output; subsequent indices denote decoder\-block outputs\. Standardization is fitted on training data, followed by a class\-balanced logistic regression classifier\. Correctness labels describe the model’s global decision\. Membership probes instead predict whether a specified candidate belongs to the valid solution, fitting a separate classifier for each candidate identifier\. These targets measure different properties of the same global\-prompt representation\.

Nested evaluation uses five outer folds and four inner folds\. The inner folds select the layer; outer folds evaluate the selected probe on held\-out instances\. Repeated evaluations use seeds 13, 23, 37, 51, and 79\. Membership scores average candidate\-specific AUCs\. Constant\-label groups are omitted, and class counts constrain the available folds\. Single\-layer curves separately describe the distribution of decodable information across depth; their highest points are layer\-search summaries rather than nested estimates\.

#### Clean\-state replacement

Intervention recipients are instances answered correctly under the clean input and incorrectly after modification of a non\-solution card\. The modified evidence preserves the enumerated valid solution\. A corresponding donor supplies states from the same clean instance; a mismatched donor supplies states from another clean instance with the required input alignment\. The evaluated mismatched donors have different target solutions\. Each comparison holds the recipient and intervention location fixed\.

Replacement acts on decoder\-block outputs at prompt positions during generation\. Image\-token interventions replace image positions; text\-token interventions replace the complementary prompt positions; full\-prompt interventions replace both\. These position sets can carry shared information after attention\. Their comparison measures which positions support recovery when replaced\. Generated\-token positions are not directly replaced\.

#### Uncertainty and depth

The paired donor contrast is the mean difference in recipient validity between corresponding and mismatched replacement\. Supplementary intervals use 2,000 bootstrap resamples of recipient pairs and pointwise 95% bounds\. Repeated\-seed probe variation describes sensitivity to the evaluation split\. Training\-label shuffles are reported separately because they hold the previously selected layer fixed\. The deepest tested block with a positive lower bound summarizes the tested recovery range\. Its resolution depends on the sampled blocks, and its uncertainty depends on cohort size\. Comparisons across models describe their respective selected cohorts\.

### A\.4Representative Prompts

The following examples are taken from the final task manifests\. They show the requested outputs and the distinction between local recovery, retrieval, selection, and path queries\.

#### Selection query

You are given 4 item cards\. Choose exactly 3 items satisfying all constraints:Available item IDs: Item A, Item B, Item C, Item D\- total cost <= 26\- total weight <= 16\- at least one selected item is yellow\- no more than 1 selected item may be type alpha\- all selected items must be mutually compatibleReturn only a JSON object with one key named items\. The value must be a list of exactly 3 selected item IDs from the available item IDs\.

#### Single\-item query

Look at the image of Item A\. What are its cost, weight, value, color, type, and compatible item list? Return JSON only\.

#### Symbolic evidence

Item A: cost=12, weight=4, value=6, color=green, type=gamma, compatible\_with=\[’Item B’, ’Item D’\]Item B: cost=13, weight=8, value=11, color=yellow, type=gamma, compatible\_with=\[’Item A’, ’Item C’, ’Item D’\]Item C: cost=3, weight=1, value=8, color=green, type=gamma, compatible\_with=\[’Item B’, ’Item D’\]Item D: cost=8, weight=6, value=24, color=yellow, type=alpha, compatible\_with=\[’Item A’, ’Item B’, ’Item C’\]

#### Matched retrieval query

You are given 4 item cards\. Locate the one card matching each descriptor below\. Each descriptor identifies exactly one card; do not apply the joint selection constraints\.\- Descriptor 1: cost=13\- Descriptor 2: cost=3\- Descriptor 3: cost=8Return only a JSON object with one key named items\. The value must contain exactly 3 item IDs, one for each descriptor\.

#### Compatibility\-only query

You are given 4 item cards\. Choose exactly 3 items satisfying the compatibility constraint set\.Available item IDs: Item A, Item B, Item C, Item D\- all selected items must be mutually compatibleReturn only a JSON object with one key named items\. The value must be a list of exactly 3 selected item IDs\.

#### Path query

The cards describe directed weighted edges\. Find the minimum\-cost path from A to H\. Combine edge evidence as needed\. Return only a JSON object with key path, whose value is the ordered list of node labels from start to goal\.

#### Edge query

Read Edge 03\. Return JSON with source, target, and cost only\.

## Appendix BExtended Behavioral Results

### B\.1Candidate Count and Model Scale

The main cross\-model panel emphasizes the largest candidate count\. Table[A2](https://arxiv.org/html/2610.33764#A2.T2)supplies the remaining counts, while Table[A3](https://arxiv.org/html/2610.33764#A2.T3)separates the nominal generation settings\. Together these breakdowns expose variation hidden by aggregate accuracy\. The subgroups reuse the final predictions and retain the same evaluation rule\.

Table A2:Global selection accuracy across candidate counts\.Entries report the percentage of answers satisfying all selection constraints on 400 instances per candidate count\. V and T denote visual\-grid and text\-oracle inputs, respectively\. Thek=8k=8results appear in the main cross\-model figure\.ModelInputk=2k=2k=4k=4k=6k=6Q3V99\.7553\.0018\.00Q3T100\.0067\.7529\.50Q7V100\.0076\.0029\.50Q7T100\.0083\.0036\.25Q32V100\.0090\.5039\.25Q32T100\.0086\.5049\.25G4V100\.0052\.5015\.75G4T100\.0073\.5032\.75G12V100\.0071\.7523\.50G12T100\.0060\.7517\.75G27V100\.0070\.5033\.75G27T100\.0085\.2548\.00I8V100\.0076\.0029\.50I8T100\.0084\.7549\.00L7V99\.0043\.0011\.00L7T100\.0080\.0034\.00Table A3:Final selection predictions stratified by nominal conflictccand distractor ratiodd\. Each cell is validity \(%\) over 100 instances\. These are disjoint subgroups of the main benchmark\. Parameter labels describe generation settings; the realized number of distractors is capped by the number of non\-solution items\.ModelInputkkc=0,d=0c=0,d=00,\.250,\.251,01,01,\.251,\.25Q3V441\.0071\.0039\.0061\.00Q3V617\.0030\.0010\.0015\.00Q3V810\.004\.003\.006\.00Q3T463\.0074\.0064\.0070\.00Q3T630\.0036\.0022\.0030\.00Q3T814\.0015\.0011\.0022\.00Q7V469\.0082\.0070\.0083\.00Q7V626\.0049\.0017\.0026\.00Q7V816\.0013\.0010\.0010\.00Q7T474\.0089\.0087\.0082\.00Q7T631\.0051\.0028\.0035\.00Q7T818\.0024\.0015\.0020\.00Q32V481\.0099\.0085\.0097\.00Q32V632\.0062\.0026\.0037\.00Q32V820\.0025\.0011\.0018\.00Q32T477\.0094\.0081\.0094\.00Q32T646\.0058\.0034\.0059\.00Q32T827\.0029\.0020\.0037\.00G4V462\.0063\.0033\.0052\.00G4V616\.0019\.0015\.0013\.00G4V811\.0014\.009\.007\.00G4T472\.0082\.0064\.0076\.00G4T630\.0042\.0026\.0033\.00G4T825\.0019\.005\.0017\.00G12V467\.0084\.0047\.0089\.00G12V623\.0033\.0020\.0018\.00G12V819\.0017\.008\.006\.00G12T455\.0090\.0015\.0083\.00G12T614\.0037\.0011\.009\.00G12T815\.0014\.004\.006\.00G27V470\.0081\.0053\.0078\.00G27V632\.0042\.0032\.0029\.00G27V820\.0022\.0018\.0019\.00G27T479\.0087\.0090\.0085\.00G27T643\.0051\.0046\.0052\.00G27T831\.0032\.0029\.0042\.00I8V478\.0090\.0050\.0086\.00I8V622\.0053\.0017\.0026\.00I8V817\.0020\.009\.0010\.00I8T478\.0086\.0092\.0083\.00I8T644\.0052\.0050\.0050\.00I8T832\.0040\.0021\.0037\.00L7V451\.0047\.0030\.0044\.00L7V612\.0021\.006\.005\.00L7V86\.006\.004\.003\.00L7T475\.0083\.0081\.0081\.00L7T628\.0038\.0033\.0037\.00L7T817\.0017\.0013\.0014\.00
### B\.2Local Recovery and Conditional Decisions

Field\-level scores identify which attributes contribute to local errors\. Conditional results then restrict global evaluation to instances with complete local recovery\. Denominators matter: models with different local accuracy can be evaluated on different retained subsets\. Same\-grid field breakdowns complement the main conditional\-grid figure and make its local\-recovery condition explicit\.

Table A4:Single\-card field recovery \(%\) over 8,000 item queries per model\. All denotes that all six fields of an item are correct\. This is an item\-level metric; conditional selection uses all items in an instance\.ModelCostWeightValueColorTypeCompat\.AllQ3100\.00100\.00100\.00100\.00100\.00100\.00100\.00Q7100\.00100\.00100\.00100\.00100\.00100\.00100\.00Q32100\.00100\.00100\.00100\.00100\.00100\.00100\.00G4100\.00100\.00100\.00100\.00100\.00100\.00100\.00G12100\.00100\.00100\.00100\.00100\.00100\.00100\.00G27100\.00100\.00100\.00100\.00100\.00100\.00100\.00I899\.9999\.9999\.9999\.9999\.9999\.9999\.99L7100\.00100\.00100\.00100\.00100\.00100\.00100\.00Table A5:Selection validity conditioned on all single\-card local questions being correct\.nnis the retained number of instances out of 400\. V/T are separate global runs conditioned on the same local\-query subset\. Unlike the main same\-grid control, these local questions show one card at a time\.ModelkknnV \(%\)T \(%\)Q3240099\.75100\.00Q3440053\.0067\.75Q3640018\.0029\.50Q384005\.7515\.50Q72400100\.00100\.00Q7440076\.0083\.00Q7640029\.5036\.25Q7840012\.2519\.25Q322400100\.00100\.00Q32440090\.5086\.50Q32640039\.2549\.25Q32840018\.5028\.25G42400100\.00100\.00G4440052\.5073\.50G4640015\.7532\.75G4840010\.2516\.50G122400100\.00100\.00G12440071\.7560\.75G12640023\.5017\.75G12840012\.509\.75G272400100\.00100\.00G27440070\.5085\.25G27640033\.7548\.00G27840019\.7533\.50I82400100\.00100\.00I8440076\.0084\.75I8640029\.5049\.00I8839914\.0432\.58L7240099\.00100\.00L7440043\.0080\.00L7640011\.0034\.00L784004\.7515\.25Table A6:Same\-grid local field recovery \(%\)\. All denotes all fields of one queried item\. Each row contains400​k400klocal queries\. The main\-text heatmap already reports instance\-level retention and conditional global validity\.ModelkkCostWeightValueColorTypeCompat\.AllQ32100\.00100\.00100\.00100\.00100\.00100\.00100\.00Q34100\.00100\.00100\.00100\.00100\.00100\.00100\.00Q36100\.0099\.9699\.9699\.96100\.0098\.9698\.96Q38100\.0099\.91100\.00100\.00100\.0098\.8898\.78G42100\.00100\.00100\.00100\.00100\.00100\.00100\.00G4498\.8199\.6999\.81100\.00100\.0098\.4497\.19G4699\.9699\.8398\.42100\.00100\.0094\.3392\.83G4899\.9495\.7599\.1299\.9198\.4493\.1287\.16I82100\.00100\.00100\.00100\.00100\.00100\.00100\.00I84100\.00100\.00100\.00100\.00100\.0099\.1299\.12I86100\.00100\.00100\.00100\.00100\.0099\.2999\.29I88100\.0099\.94100\.00100\.0099\.9799\.7599\.66L72100\.00100\.00100\.00100\.00100\.00100\.00100\.00L74100\.0099\.94100\.00100\.00100\.0095\.7595\.69L76100\.0099\.9699\.6299\.8899\.5473\.9273\.33L78100\.0099\.4799\.3499\.1998\.8181\.1679\.59
### B\.3Retrieval, Constraint Requirements, and Paths

The main diagnostic panels focus on eight candidates\. The following results supply smaller candidate counts for both modalities\. Each relaxed\-constraint score uses its own enumerated acceptance set, allowing readers to interpret the performance changes alongside the solution\-count table\. Path supplements provide the text condition at smaller graph sizes and separate response parsing from exact\-path correctness\. High parsing rates locate the remaining failures in the predicted answer rather than an absent response\.

Table A7:Exact target\-set retrieval \(%\) for the remaining candidate counts\. Each cell uses 400 instances; the main text reportsk=8k=8\.ModelInputk=2k=2k=4k=4k=6k=6Q3V100\.0087\.2589\.75Q3T100\.0083\.0083\.50Q7V100\.00100\.0099\.75Q7T100\.00100\.0098\.25G4V100\.0087\.7582\.75G4T100\.0098\.5090\.75G12V100\.0099\.7598\.50G12T100\.00100\.0099\.50Table A8:Constraint\-ablation validity \(%\) at the remaining candidate counts, 400 instances per cell\. Full is the within\-suite prompt control; Num\., Cat\., and Rel\. retain numerical, categorical, and compatibility requirements, respectively\. Required subset size is retained throughout\.ModelInputkkFullNum\.Cat\.Rel\.Q3V299\.50100\.0099\.00100\.00Q3V452\.0072\.0097\.2561\.75Q3V618\.2546\.2596\.7522\.25Q3T2100\.00100\.00100\.00100\.00Q3T469\.5079\.7596\.5081\.50Q3T631\.5046\.5095\.7550\.50Q7V2100\.00100\.00100\.00100\.00Q7V474\.5086\.2599\.0088\.50Q7V630\.2564\.5097\.5049\.50Q7T2100\.00100\.00100\.00100\.00Q7T482\.5082\.2599\.0087\.75Q7T640\.0050\.0094\.7552\.75G4V2100\.00100\.00100\.00100\.00G4V456\.5064\.7598\.5082\.75G4V616\.0045\.7594\.0033\.00G4T2100\.00100\.00100\.00100\.00G4T472\.0077\.7598\.2580\.25G4T633\.5051\.7594\.7549\.50G12V2100\.00100\.00100\.00100\.00G12V471\.7580\.0098\.7586\.75G12V626\.2552\.2597\.2558\.75G12T2100\.00100\.00100\.00100\.00G12T460\.5077\.2596\.7580\.50G12T619\.2562\.5095\.7564\.75Table A9:Text\-oracle exact\-path accuracy \(%\) at the remaining graph sizes, 400 instances per cell\. The main text provides the full visual curve and the sixteen\-edge text result\.Model4 edges8 edges12 edgesQ396\.2556\.0041\.50Q799\.0083\.0060\.75G496\.0077\.0056\.75G1299\.2588\.2580\.75Table A10:Nonempty parsed\-path rates \(%\)\.ModelInput4 edges8 edges12 edges16 edgesQ3V100\.00100\.00100\.00100\.00Q3T100\.00100\.00100\.00100\.00Q7V100\.00100\.00100\.00100\.00Q7T100\.00100\.00100\.00100\.00G4V100\.00100\.00100\.00100\.00G4T100\.00100\.00100\.00100\.00G12V100\.00100\.00100\.00100\.00G12T100\.00100\.00100\.00100\.00Table A11:Local edge\-field accuracy \(%\) over 16,000 isolated queries\. The main\-text all\-field score is not repeated\.ModelSourceTargetCostQ3100\.00100\.00100\.00Q7100\.00100\.00100\.00G4100\.00100\.00100\.00G12100\.00100\.00100\.00

## Appendix CExtended Probing Results

### C\.1Information Across Layers

The following curves distinguish pooled correctness prediction from prediction within a fixed candidate count\. Pooled labels covary with task difficulty, so the within\-count curves provide the more focused view of decision\-related information\. Both are included to make that contrast inspectable\. The membership analyses below address solution content directly\.

Figure A1:Qwen2\.5\-VL\-3B: fixed\-layer correctness AUC, pooled and grouped by candidate count\. V/T denote visual/text input\. Each layer is independently evaluated by cross\-validation; choosing a peak from these curves is exploratory, unlike nested layer selection\. Hidden\-state index 0 denotes the embedding output\.Figure A2:Qwen2\.5\-VL\-7B: fixed\-layer correctness AUC, pooled and grouped by candidate count\. V/T denote visual/text input\. Each layer is independently evaluated by cross\-validation; choosing a peak from these curves is exploratory, unlike nested layer selection\. Hidden\-state index 0 denotes the embedding output\. Only the reconciled text condition is included\.Figure A3:Gemma\-3\-4B: fixed\-layer correctness AUC, pooled and grouped by candidate count\. V/T denote visual/text input\. Each layer is independently evaluated by cross\-validation; choosing a peak from these curves is exploratory, unlike nested layer selection\. Hidden\-state index 0 denotes the embedding output\.Figure A4:Gemma\-3\-12B: fixed\-layer correctness AUC, pooled and grouped by candidate count\. V/T denote visual/text input\. Each layer is independently evaluated by cross\-validation; choosing a peak from these curves is exploratory, unlike nested layer selection\. Hidden\-state index 0 denotes the embedding output\.Figure A5:InternVL3\-8B: fixed\-layer correctness AUC, pooled and grouped by candidate count\. V/T denote visual/text input\. Each layer is independently evaluated by cross\-validation; choosing a peak from these curves is exploratory, unlike nested layer selection\. Hidden\-state index 0 denotes the embedding output\.Figure A6:LLaVA\-OneVision\-7B: fixed\-layer correctness AUC, pooled and grouped by candidate count\. V/T denote visual/text input\. Each layer is independently evaluated by cross\-validation; choosing a peak from these curves is exploratory, unlike nested layer selection\. Hidden\-state index 0 denotes the embedding output\.
### C\.2Additional Models and Label Controls

The additional\-model nested results extend the main repeated\-probe comparison to InternVL3 and LLaVA\-OneVision\. Label\-shuffle scores characterize the reference behavior under disrupted training labels at the selected layer\. Class\-count tables explain groups omitted from the probe outputs\. These controls are reported with their actual evaluation procedure rather than converted into permutation significance claims\.

Table A12:Additional\-model nested correctness probes\. AUC and accuracy are means across outer folds\. Positive rate describes class balance; depth is the mean selected hidden\-state index normalized by the final index\. These models are absent from the main repeated\-probe figure\.ModelInputkkAUCAcc\.PositiveDepthI8V40\.7900\.7200\.7600\.157I8V60\.5610\.6100\.2950\.443I8V80\.4700\.7350\.1400\.557I8T40\.8730\.8500\.8480\.457I8T60\.6210\.5880\.4900\.386I8T80\.6160\.6200\.3250\.836L7V40\.7030\.6450\.4300\.629L7V60\.7130\.7850\.1100\.479L7V80\.5960\.9030\.0480\.664L7T40\.7920\.7750\.8000\.700L7T60\.5670\.5570\.3400\.564L7T80\.5650\.7280\.1520\.800Table A13:Training\-label shuffle controls at the already selected layer\. SD is across pooled shuffle evaluations, not a confidence interval or a nested permutation test\. Each included run uses five outer folds and ten shuffles per fold\. The main repeated\-seed runs use zero shuffles\.ModelInputkkShuffle AUCSDShuffles/foldQ3V40\.5050\.05910Q3V60\.4860\.08110Q3V80\.5150\.13410Q3T40\.5160\.08910Q3T60\.5220\.08310Q3T80\.5010\.10210Q7T40\.5110\.09510Q7T60\.5090\.06110Q7T80\.5080\.10010G4V40\.4950\.06710G4V60\.5190\.08510G4V80\.4920\.10110G4T40\.4940\.07310G4T60\.4940\.07410G4T80\.5070\.10310G12V40\.5270\.07710G12V60\.5060\.08510G12V80\.5120\.10510G12T40\.5260\.07510G12T60\.4960\.07910G12T80\.5050\.13310I8V40\.5040\.07410I8V60\.5070\.07010I8V80\.5050\.08610I8T40\.5290\.09610I8T60\.5090\.08110I8T80\.5100\.06210L7V40\.5150\.07510L7V60\.4790\.11010L7V80\.5050\.14810L7T40\.4940\.08610L7T60\.4850\.06710L7T80\.5010\.09810Table A14:Candidate\-count groups absent from the fixed\-layer five\-fold probe outputs\. Positive counts refer to the final behavioral predictions; all remaining instances are negatives\.ModelInputkkPositivennQ3V2399400Q3T2400400Q7V2400400Q7T2400400G4V2400400G4T2400400G12V2400400G12T2400400I8V2400400I8T2400400L7T2400400Table A15:LLaVA\-OneVision visual correctness atk=2k=2: all recorded fixed\-layer scores\. There are 396 positive and four negative instances\. These small\-minority\-class results are reported separately from the five\-outer\-fold nested analysis\. SD describes the recorded cross\-validation folds\.IndexAUCAUC SDAccuracy00\.5000\.0000\.01010\.7850\.1590\.98520\.8540\.0250\.98530\.8810\.0310\.98340\.9370\.0230\.98250\.9700\.0100\.98260\.9670\.0150\.98070\.9670\.0230\.98080\.9620\.0270\.99090\.9620\.0150\.982100\.9390\.0420\.985110\.9220\.0500\.982120\.9240\.0270\.982130\.9170\.0190\.983140\.9490\.0330\.985150\.9670\.0180\.980160\.9800\.0120\.982170\.9720\.0040\.985180\.9850\.0050\.988190\.9920\.0080\.988200\.9970\.0040\.988210\.9970\.0040\.995220\.9970\.0040\.998230\.9970\.0040\.995240\.9970\.0040\.992250\.9970\.0040\.992260\.9970\.0040\.992270\.9970\.0040\.992280\.9970\.0040\.995
### C\.3Split Sensitivity and Candidate Membership

The main probe figure reports seed\-level means and standard deviations\. The tables below expose individual seed scores, candidate\-specific membership results, and outer\-fold layer choices\. They show whether an aggregate pattern is shared across candidates and whether the selected layer is stable across splits\. Membership decoding answers whether solution membership can be recovered from the state; correctness decoding answers whether that state predicts the model’s eventual success\.

Table A16:Qwen2\.5\-VL\-3B: per\-seed held\-out correctness AUC\. The main text presents mean and SD; this table exposes sensitivity to the split without reprinting that summary\.ModelInputkk1323375179Q3V40\.7540\.7540\.7140\.7400\.759Q3V60\.5690\.5420\.5080\.5410\.581Q3V80\.5940\.5980\.5470\.5430\.598Q3T40\.7920\.8010\.7870\.7890\.792Q3T60\.6060\.6280\.6640\.6690\.588Q3T80\.6150\.5440\.5880\.5970\.559Table A17:Qwen2\.5\-VL\-7B: per\-seed held\-out correctness AUC\. The main text presents mean and SD; this table exposes sensitivity to the split without reprinting that summary\.ModelInputkk1323375179Q7T40\.8210\.8150\.8510\.8290\.810Q7T60\.5990\.6070\.5880\.5250\.603Q7T80\.5650\.5570\.6350\.5760\.548Table A18:Gemma\-3\-4B: per\-seed held\-out correctness AUC\. The main text presents mean and SD; this table exposes sensitivity to the split without reprinting that summary\.ModelInputkk1323375179G4V40\.6450\.6420\.6820\.6400\.670G4V60\.5850\.5750\.6440\.6130\.585G4V80\.4580\.5020\.5880\.5710\.484G4T40\.7650\.7530\.7780\.7750\.782G4T60\.6570\.6460\.6530\.6680\.630G4T80\.5830\.5930\.6120\.6230\.592Table A19:Gemma\-3\-12B: per\-seed held\-out correctness AUC\. The main text presents mean and SD; this table exposes sensitivity to the split without reprinting that summary\.ModelInputkk1323375179G12V40\.7980\.7700\.7780\.7660\.744G12V60\.5240\.5310\.5870\.5640\.545G12V80\.4510\.5490\.5250\.5090\.573G12T40\.8930\.8830\.8700\.8790\.851G12T60\.6440\.6200\.5580\.6100\.611G12T80\.5370\.5870\.5020\.5610\.556Table A20:Qwen2\.5\-VL\-3B: candidate\-specific membership AUC for each seed\. A–H denote candidate IDs\. Positive is the fraction of instances whose valid subset contains that candidate\. The main figure averages these candidate\-level results\.InputkkID1323375179PositiveV4A0\.9930\.9980\.9950\.9980\.9980\.792V4B0\.9670\.9780\.9800\.9690\.9730\.745V4C0\.9670\.9700\.9600\.9670\.9790\.755V4D0\.9760\.9910\.9940\.9900\.9850\.708V6A0\.9490\.9590\.9450\.9610\.9550\.500V6B0\.8450\.8380\.8620\.8460\.8290\.510V6C0\.8490\.7780\.8030\.8170\.8060\.495V6D0\.7720\.7870\.7920\.7610\.7400\.515V6E0\.7630\.7470\.7470\.7450\.7470\.470V6F0\.7850\.7940\.7920\.7800\.8080\.510V8A0\.8510\.8280\.8440\.8710\.8500\.420V8B0\.7320\.7400\.7290\.7200\.7190\.345V8C0\.6770\.6790\.7020\.6770\.6840\.375V8D0\.6640\.6540\.6770\.6150\.6460\.338V8E0\.6110\.6090\.6510\.6110\.6290\.388V8F0\.6090\.6210\.6080\.6180\.6510\.357V8G0\.7170\.6890\.6840\.6940\.6490\.412V8H0\.6510\.6330\.6060\.6560\.6500\.365T4A0\.9980\.9950\.9950\.9980\.9980\.792T4B0\.9750\.9850\.9850\.9820\.9880\.745T4C0\.9750\.9930\.9860\.9900\.9900\.755T4D0\.9910\.9930\.9930\.9930\.9880\.708T6A0\.9540\.9550\.9580\.9620\.9560\.500T6B0\.8870\.8950\.8970\.9150\.8930\.510T6C0\.8240\.8470\.8160\.8270\.8430\.495T6D0\.7640\.7880\.7630\.7870\.7740\.515T6E0\.8220\.8160\.8130\.8210\.8440\.470T6F0\.8610\.8650\.8610\.8570\.8670\.510T8A0\.8560\.8470\.8690\.8710\.8870\.420T8B0\.7320\.7360\.7250\.7420\.7310\.345T8C0\.6870\.6400\.6140\.6520\.6980\.375T8D0\.6430\.6680\.6630\.6230\.6740\.338T8E0\.6860\.6270\.6610\.6650\.6860\.388T8F0\.7120\.7140\.7080\.6770\.6810\.357T8G0\.7580\.7400\.7520\.7420\.7520\.412T8H0\.8010\.7980\.7710\.8100\.7890\.365Table A21:Qwen2\.5\-VL\-7B: candidate\-specific membership AUC for each seed\. A–H denote candidate IDs\. Positive is the fraction of instances whose valid subset contains that candidate\. The main figure averages these candidate\-level results\.InputkkID1323375179PositiveV4A0\.9890\.9900\.9880\.9840\.9870\.792V4B0\.9820\.9840\.9830\.9830\.9890\.745V4C0\.9800\.9870\.9920\.9870\.9830\.755V4D0\.9740\.9820\.9850\.9850\.9750\.708V6A0\.9700\.9680\.9650\.9570\.9640\.500V6B0\.8270\.8570\.8470\.8590\.8510\.510V6C0\.7530\.6990\.7550\.7630\.6920\.495V6D0\.7970\.7880\.7640\.7900\.7990\.515V6E0\.7570\.7540\.7550\.7430\.7530\.470V6F0\.7980\.8060\.7740\.7850\.7770\.510V8A0\.9030\.9050\.9120\.9160\.9090\.420V8B0\.8280\.8230\.8410\.8130\.8000\.345V8C0\.6830\.6760\.6630\.6180\.6630\.375V8D0\.6430\.6230\.6380\.6540\.6580\.338V8E0\.6180\.6280\.6310\.6270\.5950\.388V8F0\.6600\.6760\.7090\.6430\.6190\.357V8G0\.6850\.6520\.6260\.6640\.6180\.412V8H0\.7240\.7310\.7180\.7460\.7200\.365T4A0\.9940\.9950\.9940\.9910\.9950\.792T4B0\.9790\.9860\.9830\.9910\.9800\.745T4C0\.9750\.9740\.9860\.9770\.9710\.755T4D0\.9900\.9890\.9900\.9900\.9840\.708T6A0\.9750\.9750\.9810\.9770\.9730\.500T6B0\.8910\.9150\.8970\.9050\.8990\.510T6C0\.8430\.8450\.8520\.8430\.8520\.495T6D0\.7760\.7840\.7960\.7600\.7780\.515T6E0\.8260\.8470\.8460\.8540\.8260\.470T6F0\.8840\.8830\.8790\.8750\.8730\.510T8A0\.8990\.9160\.9100\.8870\.8990\.420T8B0\.7590\.7970\.7720\.7680\.7610\.345T8C0\.6930\.7120\.6640\.7280\.6660\.375T8D0\.6490\.6820\.6560\.6360\.6830\.338T8E0\.7010\.6740\.7150\.7540\.7130\.388T8F0\.7220\.6860\.7090\.7010\.7650\.357T8G0\.7160\.7190\.7050\.7130\.7120\.412T8H0\.7910\.8150\.8230\.8130\.8060\.365Table A22:Gemma\-3\-4B: candidate\-specific membership AUC for each seed\. A–H denote candidate IDs\. Positive is the fraction of instances whose valid subset contains that candidate\. The main figure averages these candidate\-level results\.InputkkID1323375179PositiveV4A0\.9920\.9910\.9920\.9810\.9950\.792V4B0\.9760\.9720\.9660\.9780\.9710\.745V4C0\.9750\.9860\.9820\.9820\.9810\.755V4D0\.9730\.9820\.9860\.9800\.9780\.708V6A0\.9270\.9160\.9280\.9160\.9200\.500V6B0\.8590\.8700\.8620\.8450\.8720\.510V6C0\.8510\.8650\.8550\.8440\.8570\.495V6D0\.8150\.7950\.7850\.8160\.8040\.515V6E0\.8320\.8250\.8290\.8190\.8300\.470V6F0\.7840\.7780\.8070\.7770\.7870\.510V8A0\.8120\.8050\.8160\.8060\.8180\.420V8B0\.6760\.6770\.6840\.6450\.6190\.345V8C0\.6750\.6490\.6610\.6920\.6840\.375V8D0\.6950\.5950\.6470\.6730\.6370\.338V8E0\.7560\.6950\.7290\.7160\.7400\.388V8F0\.6670\.6810\.6690\.6670\.6710\.357V8G0\.7450\.7250\.7360\.7430\.7290\.412V8H0\.6970\.7140\.6920\.6860\.6770\.365T4A0\.9950\.9980\.9990\.9960\.9980\.792T4B0\.9790\.9710\.9780\.9830\.9830\.745T4C0\.9690\.9640\.9840\.9820\.9850\.755T4D0\.9850\.9900\.9830\.9910\.9890\.708T6A0\.9560\.9610\.9720\.9650\.9630\.500T6B0\.9130\.9350\.9390\.9240\.9170\.510T6C0\.7800\.7950\.7680\.7950\.7840\.495T6D0\.8050\.7670\.8020\.7730\.7890\.515T6E0\.7990\.8000\.8030\.8310\.8140\.470T6F0\.8550\.8570\.8640\.8510\.8630\.510T8A0\.9250\.9310\.9130\.9330\.9280\.420T8B0\.7600\.7950\.7160\.7640\.7240\.345T8C0\.6380\.6200\.6150\.6560\.6630\.375T8D0\.5910\.6070\.6130\.5780\.6030\.338T8E0\.6040\.6190\.6270\.5920\.6360\.388T8F0\.6370\.6320\.5680\.5770\.6420\.357T8G0\.6860\.6760\.6680\.6460\.6900\.412T8H0\.7630\.7720\.7640\.7760\.7610\.365Table A23:Gemma\-3\-12B: candidate\-specific membership AUC for each seed\. A–H denote candidate IDs\. Positive is the fraction of instances whose valid subset contains that candidate\. The main figure averages these candidate\-level results\.InputkkID1323375179PositiveV4A0\.9970\.9980\.9960\.9890\.9920\.792V4B0\.9750\.9800\.9790\.9800\.9760\.745V4C0\.9670\.9680\.9700\.9720\.9650\.755V4D0\.9790\.9860\.9760\.9800\.9810\.708V6A0\.9720\.9680\.9700\.9620\.9770\.500V6B0\.8840\.9050\.8910\.8900\.8830\.510V6C0\.7900\.8090\.7610\.7910\.7740\.495V6D0\.7910\.7770\.7810\.7780\.7830\.515V6E0\.7990\.8200\.8110\.8210\.7930\.470V6F0\.8150\.7880\.7910\.8270\.7910\.510V8A0\.9190\.9170\.9250\.9210\.9270\.420V8B0\.8380\.8270\.8020\.7760\.7990\.345V8C0\.6780\.6750\.6710\.6610\.6740\.375V8D0\.7240\.6840\.6730\.7010\.6970\.338V8E0\.7070\.6730\.7010\.6660\.6880\.388V8F0\.6360\.6240\.6310\.6600\.6400\.357V8G0\.7040\.6670\.6660\.6520\.7020\.412V8H0\.7320\.7140\.7220\.7510\.7350\.365T4A0\.9990\.9900\.9990\.9970\.9970\.792T4B0\.9910\.9870\.9860\.9890\.9930\.745T4C0\.9790\.9860\.9810\.9850\.9890\.755T4D0\.9970\.9950\.9970\.9960\.9960\.708T6A0\.9830\.9730\.9810\.9790\.9870\.500T6B0\.9180\.9200\.9250\.9160\.9160\.510T6C0\.8020\.8150\.7900\.7940\.8090\.495T6D0\.7490\.7410\.7280\.7640\.7450\.515T6E0\.8510\.8200\.8310\.8310\.8480\.470T6F0\.8620\.8350\.8750\.8740\.8910\.510T8A0\.9540\.9540\.9530\.9550\.9440\.420T8B0\.7850\.7910\.7840\.7800\.7560\.345T8C0\.6640\.6770\.6690\.6680\.6790\.375T8D0\.6730\.6480\.6400\.6500\.6570\.338T8E0\.6550\.6660\.6670\.6510\.6780\.388T8F0\.6560\.5870\.5920\.5900\.5920\.357T8G0\.7990\.7930\.7760\.7950\.7820\.412T8H0\.7880\.8050\.7780\.8200\.7670\.365Table A24:Selected hidden\-state indices for the five outer folds in each correctness run\. These support interpretation of split sensitivity\.ModelInputkkSeedSelected indicesQ3V41325,28,28,27,28Q3V42328,26,26,29,22Q3V43726,29,21,28,26Q3V45126,28,22,21,21Q3V47928,25,26,28,22Q3V6132,28,28,27,5Q3V6231,32,2,20,19Q3V63728,25,19,14,24Q3V65128,5,6,28,27Q3V67928,28,28,28,23Q3V81325,29,28,34,27Q3V82328,23,27,28,15Q3V83725,24,28,17,23Q3V85123,28,25,27,25Q3V8793,17,23,25,28Q3T41317,17,28,24,28Q3T42324,24,17,24,15Q3T43717,17,34,31,24Q3T45127,24,24,17,18Q3T47917,24,27,17,14Q3T61335,36,36,7,24Q3T62331,26,22,36,20Q3T63727,22,31,25,21Q3T65134,32,28,25,32Q3T67925,31,23,22,34Q3T81329,22,23,22,27Q3T82324,31,33,21,28Q3T83722,9,30,23,22Q3T85122,24,22,32,35Q3T87922,22,31,23,35Q7T41322,19,27,18,20Q7T42314,12,19,9,9Q7T43715,16,21,9,13Q7T45115,19,19,8,15Q7T47914,10,18,8,16Q7T61328,26,26,1,22Q7T62317,1,28,10,22Q7T6374,28,20,4,19Q7T6511,13,19,23,18Q7T67922,26,7,10,21Q7T81323,23,3,23,24Q7T82323,23,23,23,6Q7T83723,23,23,23,23Q7T85123,23,24,23,6Q7T87923,13,12,23,23G4V41317,16,15,17,23G4V42317,23,23,24,15G4V43722,17,17,18,16G4V45116,15,23,16,17G4V47916,17,23,21,17G4V6135,6,13,3,5G4V62311,32,5,3,6G4V6373,5,5,5,5G4V6513,5,1,5,5G4V6795,9,5,5,11G4V81323,20,2,5,14G4V82320,20,20,7,18G4V83720,16,18,19,17G4V85116,19,14,16,18G4V87917,3,19,20,15G4T41320,21,18,16,28G4T42332,7,29,20,16G4T43728,20,15,16,19G4T45115,21,20,20,16G4T47917,17,29,20,15G4T61321,17,20,17,12G4T62311,26,26,17,16G4T63717,20,17,23,23G4T65117,16,17,17,17G4T67932,20,16,17,20G4T81320,26,21,18,22G4T82321,25,20,21,22G4T83721,11,18,24,24G4T85124,17,18,26,22G4T87923,13,18,21,13G12V41332,25,27,32,30G12V42338,25,21,32,28G12V43731,22,21,32,25G12V45132,26,30,25,28G12V47922,26,27,11,30G12V6134,18,4,45,45G12V62327,14,18,21,4G12V63747,14,14,21,28G12V65118,44,4,14,18G12V67914,18,14,14,4G12V8137,28,5,22,19G12V82323,6,21,22,19G12V83724,23,7,24,23G12V85119,4,5,23,6G12V87923,5,22,5,22G12T41327,27,27,27,26G12T42327,27,27,26,27G12T43726,27,26,26,32G12T45128,27,27,27,27G12T47941,27,23,27,26G12T61326,24,22,24,29G12T62311,28,23,22,29G12T63723,22,10,30,31G12T6512,23,31,26,26G12T67922,2,13,29,26G12T81326,17,44,7,7G12T82312,44,46,10,5G12T83748,12,5,27,44G12T85114,43,10,7,38G12T87938,12,7,17,27

## Appendix DExtended Activation\-Patching Results

### D\.1Recovery at Larger Candidate Counts

The main cross\-model figure compares four\-candidate cohorts\. The following panels supply six\- and eight\-candidate cohorts, displaying corresponding and mismatched donor rates separately\. Their paired contrasts and intervals follow in a table\. Separating the absolute rates from the contrast shows whether a change reflects corresponding\-donor recovery, mismatched\-donor success, or both\.

Figure A7:Q3: full\-prompt recovery atk=6,8k=6,8, complementing the maink=4k=4comparisons\. Both donor conditions use the same recipient pairs\. Curves show absolute valid\-answer rates\. Points denote tested blocks\.Figure A8:Q7: full\-prompt recovery atk=6,8k=6,8, complementing the maink=4k=4comparisons\. Both donor conditions use the same recipient pairs\. Curves show absolute valid\-answer rates\. Points denote tested blocks\.Figure A9:G4: full\-prompt recovery atk=6,8k=6,8, complementing the maink=4k=4comparisons\. Both donor conditions use the same recipient pairs\. Curves show absolute valid\-answer rates\. Points denote tested blocks\.Figure A10:G12: full\-prompt recovery atk=6,8k=6,8, complementing the maink=4k=4comparisons\. Both donor conditions use the same recipient pairs\. Curves show absolute valid\-answer rates\. Points denote tested blocks\.Figure A11:I8: full\-prompt recovery atk=6,8k=6,8, complementing the maink=4k=4comparisons\. Both donor conditions use the same recipient pairs\. Curves show absolute valid\-answer rates\. Points denote tested blocks\.Table A25:Full\-prompt paired donor contrasts atk=6,8k=6,8\. Intervals use 2,000 recipient\-level bootstrap resamples and are pointwise\. Cohort sizes appear in the corresponding recovery panels and main coverage table\. Boundary intervals can collapse when all observed differences are identical\.ModelkkBlockΔ\\Delta95% intervalG12601\.000\[1\.000, 1\.000\]G12691\.000\[1\.000, 1\.000\]G126190\.800\[0\.600, 1\.000\]G126260\.400\[0\.133, 0\.667\]G126310\.067\[0\.000, 0\.200\]G126330\.067\[0\.000, 0\.200\]G126350\.067\[0\.000, 0\.200\]G126380\.067\[0\.000, 0\.200\]G126470\.000\[0\.000, 0\.000\]G12800\.923\[0\.769, 1\.000\]G12890\.923\[0\.769, 1\.000\]G128190\.846\[0\.615, 1\.000\]G128260\.385\[0\.154, 0\.615\]G128310\.077\[0\.000, 0\.231\]G128330\.077\[0\.000, 0\.231\]G12835\-0\.077\[\-0\.231, 0\.000\]G128380\.077\[0\.000, 0\.231\]G128470\.000\[0\.000, 0\.000\]G4601\.000\[1\.000, 1\.000\]G4671\.000\[1\.000, 1\.000\]G46130\.875\[0\.625, 1\.000\]G46180\.875\[0\.625, 1\.000\]G46190\.875\[0\.625, 1\.000\]G4620\-0\.250\[\-0\.500, 0\.000\]G4621\-0\.250\[\-0\.503, 0\.000\]G4623\-0\.125\[\-0\.375, 0\.000\]G46250\.000\[\-0\.375, 0\.375\]G46260\.000\[\-0\.375, 0\.375\]G46280\.000\[\-0\.375, 0\.375\]G46300\.000\[0\.000, 0\.000\]G46320\.000\[0\.000, 0\.000\]G46330\.000\[0\.000, 0\.000\]G4801\.000\[1\.000, 1\.000\]G4871\.000\[1\.000, 1\.000\]G48131\.000\[1\.000, 1\.000\]G48180\.833\[0\.500, 1\.000\]G48190\.667\[0\.333, 1\.000\]G48200\.167\[0\.000, 0\.500\]G48210\.167\[0\.000, 0\.500\]G48230\.000\[0\.000, 0\.000\]G48250\.000\[0\.000, 0\.000\]G48260\.000\[0\.000, 0\.000\]G4828\-0\.167\[\-0\.500, 0\.000\]G48300\.000\[0\.000, 0\.000\]G48320\.000\[0\.000, 0\.000\]G48330\.000\[0\.000, 0\.000\]I8601\.000\[1\.000, 1\.000\]I8651\.000\[1\.000, 1\.000\]I86111\.000\[1\.000, 1\.000\]I86151\.000\[1\.000, 1\.000\]I86180\.812\[0\.625, 1\.000\]I86190\.562\[0\.312, 0\.812\]I86200\.125\[\-0\.125, 0\.375\]I86220\.125\[\-0\.125, 0\.375\]I86230\.062\[\-0\.125, 0\.250\]I86240\.000\[0\.000, 0\.000\]I86250\.000\[0\.000, 0\.000\]I86260\.000\[0\.000, 0\.000\]I86270\.000\[0\.000, 0\.000\]I8800\.909\[0\.727, 1\.000\]I8850\.909\[0\.727, 1\.000\]I88111\.000\[1\.000, 1\.000\]I88151\.000\[1\.000, 1\.000\]I88180\.727\[0\.455, 1\.000\]I88190\.273\[\-0\.182, 0\.727\]I88200\.000\[\-0\.455, 0\.455\]I8822\-0\.273\[\-0\.545, 0\.000\]I8823\-0\.182\[\-0\.455, 0\.000\]I88240\.000\[0\.000, 0\.000\]I88250\.000\[0\.000, 0\.000\]I88260\.000\[0\.000, 0\.000\]I88270\.000\[0\.000, 0\.000\]Q3600\.930\[0\.880, 0\.970\]Q3630\.940\[0\.890, 0\.980\]Q3660\.950\[0\.900, 0\.990\]Q3690\.950\[0\.900, 0\.990\]Q36120\.940\[0\.890, 0\.980\]Q36150\.940\[0\.890, 0\.980\]Q36180\.960\[0\.920, 0\.990\]Q36210\.860\[0\.790, 0\.920\]Q36240\.640\[0\.550, 0\.730\]Q36270\.030\[\-0\.040, 0\.100\]Q3630\-0\.010\[\-0\.070, 0\.050\]Q36330\.020\[0\.000, 0\.050\]Q36350\.000\[0\.000, 0\.000\]Q3800\.900\[0\.840, 0\.950\]Q3830\.910\[0\.850, 0\.960\]Q3860\.900\[0\.840, 0\.950\]Q3890\.940\[0\.890, 0\.980\]Q38120\.910\[0\.850, 0\.960\]Q38150\.930\[0\.880, 0\.970\]Q38180\.890\[0\.820, 0\.950\]Q38210\.890\[0\.830, 0\.950\]Q38240\.670\[0\.580, 0\.760\]Q38270\.040\[\-0\.040, 0\.120\]Q38300\.010\[\-0\.060, 0\.080\]Q38330\.000\[0\.000, 0\.000\]Q38350\.000\[0\.000, 0\.000\]Q7601\.000\[1\.000, 1\.000\]Q7650\.933\[0\.800, 1\.000\]Q76111\.000\[1\.000, 1\.000\]Q76150\.933\[0\.800, 1\.000\]Q76180\.733\[0\.533, 0\.933\]Q76190\.333\[0\.133, 0\.600\]Q76200\.067\[0\.000, 0\.200\]Q7622\-0\.067\[\-0\.333, 0\.200\]Q76270\.000\[0\.000, 0\.000\]Q7801\.000\[1\.000, 1\.000\]Q7851\.000\[1\.000, 1\.000\]Q78110\.900\[0\.700, 1\.000\]Q78150\.900\[0\.700, 1\.000\]Q78180\.700\[0\.400, 1\.000\]Q78190\.500\[0\.200, 0\.800\]Q78200\.100\[0\.000, 0\.300\]Q78220\.100\[0\.000, 0\.300\]Q78270\.000\[0\.000, 0\.000\]
### D\.2Token Positions and Changed\-Item Selection

The six\-candidate scope comparison completes the main four\- and eight\-candidate panels\. Restricted scopes use their shared eight tested blocks; full\-prompt replacement additionally supplies the denser depth grid\. Changed\-item avoidance provides a complementary outcome: rejecting the modified item can accompany recovery, but a valid answer must also satisfy the remaining constraints\. The corresponding table therefore reports avoidance separately from validity\.

Figure A12:Qwen\-3B token\-scope contrasts atk=6k=6on 100 matched recipient pairs\. Shading shows pointwise 95% paired\-bootstrap intervals \(2,000 resamples\)\. Full prompt has 13 tested blocks; each restricted scope has eight\. The main text reports thek=4,8k=4,8scope comparisons\.Table A26:Decoy avoidance at Qwen\-3B block 24\. Each value is the fraction of patched outputs that omit the modified non\-solution item\. An output can avoid the decoy yet fail other constraints\.ScopekkMatchedMismatchedFull40\.8790\.000Full60\.9300\.530Full80\.9200\.630Image40\.4550\.081Image60\.7900\.500Image80\.7700\.600Text40\.4440\.081Text60\.5100\.420Text80\.6200\.490
### D\.3Selection of Intervention Cohorts

The clean/corrupt outcome counts show the pools from which recipients were selected\. The main intervention\-coverage table reports evaluated recipients and tested blocks; the table below instead reports selection outcomes before intervention\. Recovery rates apply to these clean\-correct/corrupt\-wrong cohorts\. Model differences in cohort composition and sample size are part of their interpretation, especially at larger candidate counts\.

Table A27:Pair\-bank outcome counts before intervention subsampling\. CC/CW/WC/WW refer to clean and corrupted correctness, respectively\. Only CW pairs are eligible\. Qwen\-3B uses a larger base bank plus independent extensions; eligible counts can exceed the evaluated counts in the main table\.ModelBankkkCCCWWCWWQ3base429614578481Q3base68754461013Q3ext168575471193Q3base85576412828Q3ext183542171706Q7base42737630Q7base6515575Q7base8610678G4base42834236G4base688480G4base856188G12base42047033G12base6716374G12base8613081I8base41860121I8base6616771I8base8611281

## Appendix EUncertainty in Behavioral Results

We supplement the behavioral point estimates with pointwise 95% Wilson score intervals\. Each table reports the observed number of successes and its denominator; percentages and interval endpoints are rounded to two decimal places\. These intervals quantify finite\-instance uncertainty for the evaluated checkpoint and benchmark construction\. They do not include variation across model training runs or alternative task generators\. The binomial score intervals are marginal approximations for the benchmark’s fixed mixture of generation settings\.

For unconditional selection, retrieval, constraint ablation, and path finding, the unit is one generated instance, with 400 instances at each task size\. For conditional selection, the denominator is the number of instances on which every constituent local answer is correct\. The interval therefore describes validity within that retained cohort\. For local recovery, we report the probability that all queried fields of all items in an instance are correct; grouping queries this way preserves the instance as the sampling unit\. In particular, the graph\-level intervals complement the perfect isolated\-edge scores in Table[1](https://arxiv.org/html/2610.33764#S4.T1)without treating 16,000 correlated edge queries as independent trials\.

The intervals cover individual rates, rather than paired differences between conditions, and are not adjusted for multiple comparisons\. Comparisons involving shared instances require paired analysis; overlap or non\-overlap of the marginal intervals alone is not a significance test\. Probing variability across seeds and the paired\-bootstrap intervals for activation interventions retain their separate definitions in Appendix[A\.3](https://arxiv.org/html/2610.33764#A1.SS3)\.

Throughout these tables, Q3/Q7/Q32 denote Qwen2\.5\-VL\-3B/7B/32B, G4/G12/G27 denote Gemma\-3\-4B/12B/27B, I8 denotes InternVL3\-8B, and L7 denotes LLaVA\-OneVision\-7B\. V/T denote visual\-grid/text\-oracle input\. These are uncertainty summaries of the final experiments, not additional inference runs\.

Table A28:Selection validity with 95% Wilson intervals\. Each row uses 400 final benchmark instances\.ModelInputkkCorrect/nnValidity \[95% CI\]Q3V2399/40099\.75 \[98\.60, 99\.96\]Q3V4212/40053\.00 \[48\.10, 57\.84\]Q3V672/40018\.00 \[14\.55, 22\.06\]Q3V823/4005\.75 \[3\.86, 8\.48\]Q3T2400/400100\.00 \[99\.05, 100\.00\]Q3T4271/40067\.75 \[63\.02, 72\.14\]Q3T6118/40029\.50 \[25\.24, 34\.15\]Q3T862/40015\.50 \[12\.28, 19\.37\]Q7V2400/400100\.00 \[99\.05, 100\.00\]Q7V4304/40076\.00 \[71\.58, 79\.93\]Q7V6118/40029\.50 \[25\.24, 34\.15\]Q7V849/40012\.25 \[9\.39, 15\.83\]Q7T2400/400100\.00 \[99\.05, 100\.00\]Q7T4332/40083\.00 \[79\.01, 86\.36\]Q7T6145/40036\.25 \[31\.69, 41\.07\]Q7T877/40019\.25 \[15\.69, 23\.40\]Q32V2400/400100\.00 \[99\.05, 100\.00\]Q32V4362/40090\.50 \[87\.23, 93\.00\]Q32V6157/40039\.25 \[34\.59, 44\.12\]Q32V874/40018\.50 \[15\.00, 22\.60\]Q32T2400/400100\.00 \[99\.05, 100\.00\]Q32T4346/40086\.50 \[82\.80, 89\.50\]Q32T6197/40049\.25 \[44\.38, 54\.13\]Q32T8113/40028\.25 \[24\.06, 32\.85\]G4V2400/400100\.00 \[99\.05, 100\.00\]G4V4210/40052\.50 \[47\.61, 57\.35\]G4V663/40015\.75 \[12\.51, 19\.64\]G4V841/40010\.25 \[7\.65, 13\.61\]G4T2400/400100\.00 \[99\.05, 100\.00\]G4T4294/40073\.50 \[68\.97, 77\.59\]G4T6131/40032\.75 \[28\.33, 37\.49\]G4T866/40016\.50 \[13\.18, 20\.45\]G12V2400/400100\.00 \[99\.05, 100\.00\]G12V4287/40071\.75 \[67\.15, 75\.94\]G12V694/40023\.50 \[19\.61, 27\.90\]G12V850/40012\.50 \[9\.61, 16\.10\]G12T2400/400100\.00 \[99\.05, 100\.00\]G12T4243/40060\.75 \[55\.88, 65\.41\]G12T671/40017\.75 \[14\.32, 21\.80\]G12T839/4009\.75 \[7\.21, 13\.05\]G27V2400/400100\.00 \[99\.05, 100\.00\]G27V4282/40070\.50 \[65\.85, 74\.76\]G27V6135/40033\.75 \[29\.29, 38\.52\]G27V879/40019\.75 \[16\.14, 23\.93\]G27T2400/400100\.00 \[99\.05, 100\.00\]G27T4341/40085\.25 \[81\.44, 88\.39\]G27T6192/40048\.00 \[43\.15, 52\.89\]G27T8134/40033\.50 \[29\.05, 38\.26\]I8V2400/400100\.00 \[99\.05, 100\.00\]I8V4304/40076\.00 \[71\.58, 79\.93\]I8V6118/40029\.50 \[25\.24, 34\.15\]I8V856/40014\.00 \[10\.94, 17\.74\]I8T2400/400100\.00 \[99\.05, 100\.00\]I8T4339/40084\.75 \[80\.90, 87\.94\]I8T6196/40049\.00 \[44\.13, 53\.89\]I8T8130/40032\.50 \[28\.10, 37\.24\]L7V2396/40099\.00 \[97\.46, 99\.61\]L7V4172/40043\.00 \[38\.24, 47\.90\]L7V644/40011\.00 \[8\.30, 14\.45\]L7V819/4004\.75 \[3\.06, 7\.30\]L7T2400/400100\.00 \[99\.05, 100\.00\]L7T4320/40080\.00 \[75\.80, 83\.63\]L7T6136/40034\.00 \[29\.53, 38\.77\]L7T861/40015\.25 \[12\.06, 19\.10\]Table A29:Isolated\-card complete local recovery at the instance level: every field of every item must be correct\. Denominator: 400 instances per row\.ModelkkRetained/400Recovery \[95% CI\]Q32400/400100\.00 \[99\.05, 100\.00\]Q34400/400100\.00 \[99\.05, 100\.00\]Q36400/400100\.00 \[99\.05, 100\.00\]Q38400/400100\.00 \[99\.05, 100\.00\]Q72400/400100\.00 \[99\.05, 100\.00\]Q74400/400100\.00 \[99\.05, 100\.00\]Q76400/400100\.00 \[99\.05, 100\.00\]Q78400/400100\.00 \[99\.05, 100\.00\]Q322400/400100\.00 \[99\.05, 100\.00\]Q324400/400100\.00 \[99\.05, 100\.00\]Q326400/400100\.00 \[99\.05, 100\.00\]Q328400/400100\.00 \[99\.05, 100\.00\]G42400/400100\.00 \[99\.05, 100\.00\]G44400/400100\.00 \[99\.05, 100\.00\]G46400/400100\.00 \[99\.05, 100\.00\]G48400/400100\.00 \[99\.05, 100\.00\]G122400/400100\.00 \[99\.05, 100\.00\]G124400/400100\.00 \[99\.05, 100\.00\]G126400/400100\.00 \[99\.05, 100\.00\]G128400/400100\.00 \[99\.05, 100\.00\]G272400/400100\.00 \[99\.05, 100\.00\]G274400/400100\.00 \[99\.05, 100\.00\]G276400/400100\.00 \[99\.05, 100\.00\]G278400/400100\.00 \[99\.05, 100\.00\]I82400/400100\.00 \[99\.05, 100\.00\]I84400/400100\.00 \[99\.05, 100\.00\]I86400/400100\.00 \[99\.05, 100\.00\]I88399/40099\.75 \[98\.60, 99\.96\]L72400/400100\.00 \[99\.05, 100\.00\]L74400/400100\.00 \[99\.05, 100\.00\]L76400/400100\.00 \[99\.05, 100\.00\]L78400/400100\.00 \[99\.05, 100\.00\]Table A30:Isolated\-card conditional selection validity\. The denominator is the retained cohort, not 400; these intervals describe that selected cohort\.ModelInputkkCorrect/retainedValidity \[95% CI\]Q3V2399/40099\.75 \[98\.60, 99\.96\]Q3T2400/400100\.00 \[99\.05, 100\.00\]Q3V4212/40053\.00 \[48\.10, 57\.84\]Q3T4271/40067\.75 \[63\.02, 72\.14\]Q3V672/40018\.00 \[14\.55, 22\.06\]Q3T6118/40029\.50 \[25\.24, 34\.15\]Q3V823/4005\.75 \[3\.86, 8\.48\]Q3T862/40015\.50 \[12\.28, 19\.37\]Q7V2400/400100\.00 \[99\.05, 100\.00\]Q7T2400/400100\.00 \[99\.05, 100\.00\]Q7V4304/40076\.00 \[71\.58, 79\.93\]Q7T4332/40083\.00 \[79\.01, 86\.36\]Q7V6118/40029\.50 \[25\.24, 34\.15\]Q7T6145/40036\.25 \[31\.69, 41\.07\]Q7V849/40012\.25 \[9\.39, 15\.83\]Q7T877/40019\.25 \[15\.69, 23\.40\]Q32V2400/400100\.00 \[99\.05, 100\.00\]Q32T2400/400100\.00 \[99\.05, 100\.00\]Q32V4362/40090\.50 \[87\.23, 93\.00\]Q32T4346/40086\.50 \[82\.80, 89\.50\]Q32V6157/40039\.25 \[34\.59, 44\.12\]Q32T6197/40049\.25 \[44\.38, 54\.13\]Q32V874/40018\.50 \[15\.00, 22\.60\]Q32T8113/40028\.25 \[24\.06, 32\.85\]G4V2400/400100\.00 \[99\.05, 100\.00\]G4T2400/400100\.00 \[99\.05, 100\.00\]G4V4210/40052\.50 \[47\.61, 57\.35\]G4T4294/40073\.50 \[68\.97, 77\.59\]G4V663/40015\.75 \[12\.51, 19\.64\]G4T6131/40032\.75 \[28\.33, 37\.49\]G4V841/40010\.25 \[7\.65, 13\.61\]G4T866/40016\.50 \[13\.18, 20\.45\]G12V2400/400100\.00 \[99\.05, 100\.00\]G12T2400/400100\.00 \[99\.05, 100\.00\]G12V4287/40071\.75 \[67\.15, 75\.94\]G12T4243/40060\.75 \[55\.88, 65\.41\]G12V694/40023\.50 \[19\.61, 27\.90\]G12T671/40017\.75 \[14\.32, 21\.80\]G12V850/40012\.50 \[9\.61, 16\.10\]G12T839/4009\.75 \[7\.21, 13\.05\]G27V2400/400100\.00 \[99\.05, 100\.00\]G27T2400/400100\.00 \[99\.05, 100\.00\]G27V4282/40070\.50 \[65\.85, 74\.76\]G27T4341/40085\.25 \[81\.44, 88\.39\]G27V6135/40033\.75 \[29\.29, 38\.52\]G27T6192/40048\.00 \[43\.15, 52\.89\]G27V879/40019\.75 \[16\.14, 23\.93\]G27T8134/40033\.50 \[29\.05, 38\.26\]I8V2400/400100\.00 \[99\.05, 100\.00\]I8T2400/400100\.00 \[99\.05, 100\.00\]I8V4304/40076\.00 \[71\.58, 79\.93\]I8T4339/40084\.75 \[80\.90, 87\.94\]I8V6118/40029\.50 \[25\.24, 34\.15\]I8T6196/40049\.00 \[44\.13, 53\.89\]I8V856/39914\.04 \[10\.97, 17\.79\]I8T8130/39932\.58 \[28\.17, 37\.33\]L7V2396/40099\.00 \[97\.46, 99\.61\]L7T2400/400100\.00 \[99\.05, 100\.00\]L7V4172/40043\.00 \[38\.24, 47\.90\]L7T4320/40080\.00 \[75\.80, 83\.63\]L7V644/40011\.00 \[8\.30, 14\.45\]L7T6136/40034\.00 \[29\.53, 38\.77\]L7V819/4004\.75 \[3\.06, 7\.30\]L7T861/40015\.25 \[12\.06, 19\.10\]Table A31:Same\-grid complete local recovery at the instance level: every field of every item must be correct\. Denominator: 400 instances per row\.ModelkkRetained/400Recovery \[95% CI\]Q32400/400100\.00 \[99\.05, 100\.00\]Q34400/400100\.00 \[99\.05, 100\.00\]Q36376/40094\.00 \[91\.23, 95\.94\]Q38362/40090\.50 \[87\.23, 93\.00\]G42400/400100\.00 \[99\.05, 100\.00\]G44356/40089\.00 \[85\.55, 91\.70\]G46255/40063\.75 \[58\.93, 68\.31\]G48142/40035\.50 \[30\.97, 40\.31\]I82400/400100\.00 \[99\.05, 100\.00\]I84386/40096\.50 \[94\.21, 97\.90\]I86383/40095\.75 \[93\.30, 97\.33\]I88389/40097\.25 \[95\.14, 98\.46\]L72400/400100\.00 \[99\.05, 100\.00\]L74331/40082\.75 \[78\.74, 86\.14\]L7615/4003\.75 \[2\.29, 6\.09\]L788/4002\.00 \[1\.02, 3\.90\]Table A32:Same\-grid conditional selection validity\. The denominator is the retained cohort, not 400; these intervals describe that selected cohort\.ModelInputkkCorrect/retainedValidity \[95% CI\]Q3V2399/40099\.75 \[98\.60, 99\.96\]Q3V4212/40053\.00 \[48\.10, 57\.84\]Q3V666/37617\.55 \[14\.04, 21\.72\]Q3V821/3625\.80 \[3\.83, 8\.71\]G4V2400/400100\.00 \[99\.05, 100\.00\]G4V4192/35653\.93 \[48\.74, 59\.04\]G4V637/25514\.51 \[10\.71, 19\.36\]G4V815/14210\.56 \[6\.51, 16\.70\]I8V2400/400100\.00 \[99\.05, 100\.00\]I8V4295/38676\.42 \[71\.94, 80\.39\]I8V6113/38329\.50 \[25\.16, 34\.26\]I8V852/38913\.37 \[10\.34, 17\.11\]L7V2396/40099\.00 \[97\.46, 99\.61\]L7V4141/33142\.60 \[37\.39, 47\.98\]L7V62/1513\.33 \[3\.74, 37\.88\]L7V81/812\.50 \[2\.24, 47\.09\]Table A33:Target\-set retrieval with 95% Wilson intervals; 400 instances per row\.ModelInputkkCorrect/nnRate \[95% CI\]Q3V2400/400100\.00 \[99\.05, 100\.00\]Q3V4349/40087\.25 \[83\.62, 90\.17\]Q3V6359/40089\.75 \[86\.39, 92\.35\]Q3V8341/40085\.25 \[81\.44, 88\.39\]Q3T2400/400100\.00 \[99\.05, 100\.00\]Q3T4332/40083\.00 \[79\.01, 86\.36\]Q3T6334/40083\.50 \[79\.55, 86\.82\]Q3T8320/40080\.00 \[75\.80, 83\.63\]Q7V2400/400100\.00 \[99\.05, 100\.00\]Q7V4400/400100\.00 \[99\.05, 100\.00\]Q7V6399/40099\.75 \[98\.60, 99\.96\]Q7V8390/40097\.50 \[95\.46, 98\.64\]Q7T2400/400100\.00 \[99\.05, 100\.00\]Q7T4400/400100\.00 \[99\.05, 100\.00\]Q7T6393/40098\.25 \[96\.43, 99\.15\]Q7T8377/40094\.25 \[91\.52, 96\.14\]G4V2400/400100\.00 \[99\.05, 100\.00\]G4V4351/40087\.75 \[84\.17, 90\.61\]G4V6331/40082\.75 \[78\.74, 86\.14\]G4V8263/40065\.75 \[60\.97, 70\.23\]G4T2400/400100\.00 \[99\.05, 100\.00\]G4T4394/40098\.50 \[96\.77, 99\.31\]G4T6363/40090\.75 \[87\.51, 93\.21\]G4T8305/40076\.25 \[71\.84, 80\.16\]G12V2400/400100\.00 \[99\.05, 100\.00\]G12V4399/40099\.75 \[98\.60, 99\.96\]G12V6394/40098\.50 \[96\.77, 99\.31\]G12V8389/40097\.25 \[95\.14, 98\.46\]G12T2400/400100\.00 \[99\.05, 100\.00\]G12T4400/400100\.00 \[99\.05, 100\.00\]G12T6398/40099\.50 \[98\.20, 99\.86\]G12T8392/40098\.00 \[96\.10, 98\.98\]Table A34:Constraint\-ablation validity with 95% Wilson intervals; 400 instances per row\.ModelInputkkScopeCorrect/nnRate \[95% CI\]Q3V2Full398/40099\.50 \[98\.20, 99\.86\]Q3V4Full208/40052\.00 \[47\.11, 56\.85\]Q3V6Full73/40018\.25 \[14\.77, 22\.33\]Q3V8Full24/4006\.00 \[4\.06, 8\.77\]Q3V2Num\.400/400100\.00 \[99\.05, 100\.00\]Q3V4Num\.288/40072\.00 \[67\.41, 76\.17\]Q3V6Num\.185/40046\.25 \[41\.42, 51\.15\]Q3V8Num\.157/40039\.25 \[34\.59, 44\.12\]Q3V2Cat\.396/40099\.00 \[97\.46, 99\.61\]Q3V4Cat\.389/40097\.25 \[95\.14, 98\.46\]Q3V6Cat\.387/40096\.75 \[94\.52, 98\.09\]Q3V8Cat\.361/40090\.25 \[86\.95, 92\.79\]Q3V2Rel\.400/400100\.00 \[99\.05, 100\.00\]Q3V4Rel\.247/40061\.75 \[56\.90, 66\.38\]Q3V6Rel\.89/40022\.25 \[18\.45, 26\.58\]Q3V8Rel\.47/40011\.75 \[8\.95, 15\.28\]Q3T2Full400/400100\.00 \[99\.05, 100\.00\]Q3T4Full278/40069\.50 \[64\.82, 73\.81\]Q3T6Full126/40031\.50 \[27\.14, 36\.21\]Q3T8Full65/40016\.25 \[12\.96, 20\.18\]Q3T2Num\.400/400100\.00 \[99\.05, 100\.00\]Q3T4Num\.319/40079\.75 \[75\.54, 83\.40\]Q3T6Num\.186/40046\.50 \[41\.67, 51\.40\]Q3T8Num\.132/40033\.00 \[28\.57, 37\.75\]Q3T2Cat\.400/400100\.00 \[99\.05, 100\.00\]Q3T4Cat\.386/40096\.50 \[94\.21, 97\.90\]Q3T6Cat\.383/40095\.75 \[93\.30, 97\.33\]Q3T8Cat\.360/40090\.00 \[86\.67, 92\.57\]Q3T2Rel\.400/400100\.00 \[99\.05, 100\.00\]Q3T4Rel\.326/40081\.50 \[77\.40, 85\.00\]Q3T6Rel\.202/40050\.50 \[45\.62, 55\.37\]Q3T8Rel\.157/40039\.25 \[34\.59, 44\.12\]Q7V2Full400/400100\.00 \[99\.05, 100\.00\]Q7V4Full298/40074\.50 \[70\.01, 78\.52\]Q7V6Full121/40030\.25 \[25\.95, 34\.92\]Q7V8Full53/40013\.25 \[10\.27, 16\.92\]Q7V2Num\.400/400100\.00 \[99\.05, 100\.00\]Q7V4Num\.345/40086\.25 \[82\.53, 89\.28\]Q7V6Num\.258/40064\.50 \[59\.69, 69\.03\]Q7V8Num\.198/40049\.50 \[44\.63, 54\.38\]Q7V2Cat\.400/400100\.00 \[99\.05, 100\.00\]Q7V4Cat\.396/40099\.00 \[97\.46, 99\.61\]Q7V6Cat\.390/40097\.50 \[95\.46, 98\.64\]Q7V8Cat\.381/40095\.25 \[92\.70, 96\.94\]Q7V2Rel\.400/400100\.00 \[99\.05, 100\.00\]Q7V4Rel\.354/40088\.50 \[85\.00, 91\.27\]Q7V6Rel\.198/40049\.50 \[44\.63, 54\.38\]Q7V8Rel\.119/40029\.75 \[25\.48, 34\.41\]Q7T2Full400/400100\.00 \[99\.05, 100\.00\]Q7T4Full330/40082\.50 \[78\.47, 85\.91\]Q7T6Full160/40040\.00 \[35\.32, 44\.87\]Q7T8Full83/40020\.75 \[17\.06, 24\.99\]Q7T2Num\.400/400100\.00 \[99\.05, 100\.00\]Q7T4Num\.329/40082\.25 \[78\.20, 85\.68\]Q7T6Num\.200/40050\.00 \[45\.12, 54\.88\]Q7T8Num\.170/40042\.50 \[37\.75, 47\.39\]Q7T2Cat\.400/400100\.00 \[99\.05, 100\.00\]Q7T4Cat\.396/40099\.00 \[97\.46, 99\.61\]Q7T6Cat\.379/40094\.75 \[92\.11, 96\.54\]Q7T8Cat\.377/40094\.25 \[91\.52, 96\.14\]Q7T2Rel\.400/400100\.00 \[99\.05, 100\.00\]Q7T4Rel\.351/40087\.75 \[84\.17, 90\.61\]Q7T6Rel\.211/40052\.75 \[47\.85, 57\.59\]Q7T8Rel\.166/40041\.50 \[36\.77, 46\.39\]G4V2Full400/400100\.00 \[99\.05, 100\.00\]G4V4Full226/40056\.50 \[51\.60, 61\.27\]G4V6Full64/40016\.00 \[12\.73, 19\.91\]G4V8Full47/40011\.75 \[8\.95, 15\.28\]G4V2Num\.400/400100\.00 \[99\.05, 100\.00\]G4V4Num\.259/40064\.75 \[59\.95, 69\.27\]G4V6Num\.183/40045\.75 \[40\.93, 50\.65\]G4V8Num\.149/40037\.25 \[32\.65, 42\.09\]G4V2Cat\.400/400100\.00 \[99\.05, 100\.00\]G4V4Cat\.394/40098\.50 \[96\.77, 99\.31\]G4V6Cat\.376/40094\.00 \[91\.23, 95\.94\]G4V8Cat\.364/40091\.00 \[87\.79, 93\.43\]G4V2Rel\.400/400100\.00 \[99\.05, 100\.00\]G4V4Rel\.331/40082\.75 \[78\.74, 86\.14\]G4V6Rel\.132/40033\.00 \[28\.57, 37\.75\]G4V8Rel\.56/40014\.00 \[10\.94, 17\.74\]G4T2Full400/400100\.00 \[99\.05, 100\.00\]G4T4Full288/40072\.00 \[67\.41, 76\.17\]G4T6Full134/40033\.50 \[29\.05, 38\.26\]G4T8Full61/40015\.25 \[12\.06, 19\.10\]G4T2Num\.400/400100\.00 \[99\.05, 100\.00\]G4T4Num\.311/40077\.75 \[73\.42, 81\.55\]G4T6Num\.207/40051\.75 \[46\.86, 56\.61\]G4T8Num\.192/40048\.00 \[43\.15, 52\.89\]G4T2Cat\.400/400100\.00 \[99\.05, 100\.00\]G4T4Cat\.393/40098\.25 \[96\.43, 99\.15\]G4T6Cat\.379/40094\.75 \[92\.11, 96\.54\]G4T8Cat\.356/40089\.00 \[85\.55, 91\.70\]G4T2Rel\.400/400100\.00 \[99\.05, 100\.00\]G4T4Rel\.321/40080\.25 \[76\.07, 83\.86\]G4T6Rel\.198/40049\.50 \[44\.63, 54\.38\]G4T8Rel\.148/40037\.00 \[32\.41, 41\.83\]G12V2Full400/400100\.00 \[99\.05, 100\.00\]G12V4Full287/40071\.75 \[67\.15, 75\.94\]G12V6Full105/40026\.25 \[22\.18, 30\.77\]G12V8Full57/40014\.25 \[11\.16, 18\.02\]G12V2Num\.400/400100\.00 \[99\.05, 100\.00\]G12V4Num\.320/40080\.00 \[75\.80, 83\.63\]G12V6Num\.209/40052\.25 \[47\.36, 57\.10\]G12V8Num\.223/40055\.75 \[50\.85, 60\.54\]G12V2Cat\.400/400100\.00 \[99\.05, 100\.00\]G12V4Cat\.395/40098\.75 \[97\.11, 99\.46\]G12V6Cat\.389/40097\.25 \[95\.14, 98\.46\]G12V8Cat\.383/40095\.75 \[93\.30, 97\.33\]G12V2Rel\.400/400100\.00 \[99\.05, 100\.00\]G12V4Rel\.347/40086\.75 \[83\.08, 89\.73\]G12V6Rel\.235/40058\.75 \[53\.86, 63\.47\]G12V8Rel\.192/40048\.00 \[43\.15, 52\.89\]G12T2Full400/400100\.00 \[99\.05, 100\.00\]G12T4Full242/40060\.50 \[55\.63, 65\.17\]G12T6Full77/40019\.25 \[15\.69, 23\.40\]G12T8Full46/40011\.50 \[8\.73, 15\.00\]G12T2Num\.400/400100\.00 \[99\.05, 100\.00\]G12T4Num\.309/40077\.25 \[72\.89, 81\.09\]G12T6Num\.250/40062\.50 \[57\.66, 67\.10\]G12T8Num\.257/40064\.25 \[59\.44, 68\.79\]G12T2Cat\.400/400100\.00 \[99\.05, 100\.00\]G12T4Cat\.387/40096\.75 \[94\.52, 98\.09\]G12T6Cat\.383/40095\.75 \[93\.30, 97\.33\]G12T8Cat\.374/40093\.50 \[90\.65, 95\.53\]G12T2Rel\.400/400100\.00 \[99\.05, 100\.00\]G12T4Rel\.322/40080\.50 \[76\.33, 84\.08\]G12T6Rel\.259/40064\.75 \[59\.95, 69\.27\]G12T8Rel\.233/40058\.25 \[53\.36, 62\.98\]Table A35:Exact\-path accuracy with 95% Wilson intervals; 400 instances per row\.ModelInputkkCorrect/nnRate \[95% CI\]Q3V4392/40098\.00 \[96\.10, 98\.98\]Q3V8324/40081\.00 \[76\.87, 84\.54\]Q3V12212/40053\.00 \[48\.10, 57\.84\]Q3V16156/40039\.00 \[34\.35, 43\.86\]Q3T4385/40096\.25 \[93\.91, 97\.71\]Q3T8224/40056\.00 \[51\.10, 60\.78\]Q3T12166/40041\.50 \[36\.77, 46\.39\]Q3T16111/40027\.75 \[23\.59, 32\.33\]Q7V4399/40099\.75 \[98\.60, 99\.96\]Q7V8325/40081\.25 \[77\.13, 84\.77\]Q7V12213/40053\.25 \[48\.35, 58\.09\]Q7V16170/40042\.50 \[37\.75, 47\.39\]Q7T4396/40099\.00 \[97\.46, 99\.61\]Q7T8332/40083\.00 \[79\.01, 86\.36\]Q7T12243/40060\.75 \[55\.88, 65\.41\]Q7T16182/40045\.50 \[40\.69, 50\.40\]G4V4376/40094\.00 \[91\.23, 95\.94\]G4V8193/40048\.25 \[43\.39, 53\.14\]G4V1292/40023\.00 \[19\.14, 27\.37\]G4V1653/40013\.25 \[10\.27, 16\.92\]G4T4384/40096\.00 \[93\.60, 97\.52\]G4T8308/40077\.00 \[72\.63, 80\.86\]G4T12227/40056\.75 \[51\.85, 61\.52\]G4T16172/40043\.00 \[38\.24, 47\.90\]G12V4397/40099\.25 \[97\.82, 99\.74\]G12V8338/40084\.50 \[80\.63, 87\.72\]G12V12188/40047\.00 \[42\.16, 51\.90\]G12V16162/40040\.50 \[35\.80, 45\.38\]G12T4397/40099\.25 \[97\.82, 99\.74\]G12T8353/40088\.25 \[84\.72, 91\.05\]G12T12323/40080\.75 \[76\.60, 84\.31\]G12T16285/40071\.25 \[66\.63, 75\.47\]Table A36:Complete isolated\-edge recovery per graph\. A success requires every field of every edge to be correct\. The graph, rather than an edge query, is the sampling unit\.ModelEdgesCorrect graphs/nnRecovery \[95% CI\]Q34400/400100\.00 \[99\.05, 100\.00\]Q38400/400100\.00 \[99\.05, 100\.00\]Q312400/400100\.00 \[99\.05, 100\.00\]Q316400/400100\.00 \[99\.05, 100\.00\]Q74400/400100\.00 \[99\.05, 100\.00\]Q78400/400100\.00 \[99\.05, 100\.00\]Q712400/400100\.00 \[99\.05, 100\.00\]Q716400/400100\.00 \[99\.05, 100\.00\]G44400/400100\.00 \[99\.05, 100\.00\]G48400/400100\.00 \[99\.05, 100\.00\]G412400/400100\.00 \[99\.05, 100\.00\]G416400/400100\.00 \[99\.05, 100\.00\]G124400/400100\.00 \[99\.05, 100\.00\]G128400/400100\.00 \[99\.05, 100\.00\]G1212400/400100\.00 \[99\.05, 100\.00\]G1216400/400100\.00 \[99\.05, 100\.00\]
## Appendix FRelation to the Main Conclusions

The supplementary evidence follows the same sequence as the main analysis\. Local and conditional breakdowns characterize the gap between recovering facts and producing a valid answer\. Retrieval and constraint diagnostics identify task requirements associated with that gap\. The controlled path task establishes a second setting with the same behavioral separation\. Probes then distinguish solution membership in global\-prompt states from prediction of decision correctness\. Finally, paired replacement measures how changes to those states alter failed decisions across depth and prompt positions\.

The empirical claim concerns the connection between recoverable evidence, decodable solution information, and successful decisions under these controlled tasks\. The path construction, changing acceptance sets, distinct model\-specific intervention cohorts, and finite layer grids specify the scope of that claim\. Together, the main and supplementary results motivate evaluating evidence recovery and evidence use separately, and studying the internal computations that connect them\.

## Appendix GPrompting Controls

Before introducing deterministic execution, we test whether additional instructions alone improve global decisions\. All four conditions use the same 400 visual\-grid instances for each model and candidate count\.*Direct*requests the selected item IDs in one response\.*Generic Review*asks the model to reconsider its initial answer\.*Extract–Select*\(ES\) first asks the model to transcribe every card into structured JSON and then select a subset from that transcription\.*Extract–Select–Verify*\(ESV\) adds an explicit check of the numerical, categorical, and compatibility requirements before the final answer\.

Table[A37](https://arxiv.org/html/2610.33764#A7.T37)shows that structured prompting alone produces uneven changes\. ES improves thek=4k=4result for both Qwen checkpoints, while itsk=6k=6andk=8k=8accuracy remains close to Direct\. Explicit verification produces similar results\. Generic Review helps both Gemma checkpoints in several conditions, while its effect is model\-dependent and frequently yields responses outside the required output format\. These comparisons motivate separating evidence transcription from joint execution in the Extract–Solve analysis\.

Table A37:Prompting controls produce model\-dependent changes without consistently resolving difficult global decisions\.Entries are exact globally valid answers \(%\) over 400 paired visual\-grid instances\. ES denotes Extract–Select and ESV denotes Extract–Select–Verify\.### G\.1Explicit Reasoning Budget

The preceding prompting controls vary response structure within the original checkpoints\. We next test whether additional inference\-time computation changes the difficult eight\-candidate result\. The evaluation uses 195 visual\-grid instances shared by every condition \(52 for conflict 0 and distractor ratio 0, 52 for conflict 0 and distractor ratio 0\.25, 52 for conflict 1 and distractor ratio 0, 39 for conflict 1 and distractor ratio 0\.25\)\. Qwen3\-VL\-8B\-Instruct and Qwen3\-VL\-8B\-Thinking receive the same images and constraints\. Each checkpoint is evaluated once with answer\-only decoding and once with a two\-stage protocol: the first stage reasons through every candidate and constraint, and the second stage emits the final JSON answer\. Sampling is disabled in every condition\. The paired sample design isolates the change associated with explicit reasoning within each checkpoint\.

Table A38:Explicit reasoning changes global validity on paired eight\-candidate visual instances\.Valid reports exact globally valid answers with pointwise Wilson 95% intervals\. Final JSON is the share of responses containing a parseable item\-set object\. At cap is the share whose final generation reaches its token budget\.Figure A13:Explicit reasoning repairs some decisions and can also overturn valid answers\.Each horizontal bar partitions the same 195 instances by their paired answer\-only and explicit\-reasoning outcomes\. Repaired denotes an invalid answer that becomes globally valid; regressed denotes a valid answer that becomes invalid\. Qwen3\-VL\-8B\-Instruct and Qwen3\-VL\-8B\-Thinking use identical instances and deterministic decoding\.#### What Additional Computation Changes

Qwen3\-VL\-8B Instruct moves from 5\.64% with answer\-only decoding to 48\.21% with explicit reasoning\. The paired gain is \+42\.56 percentage points, with a 95% interval of \[\+34\.87, \+50\.26\]\. Figure[A13](https://arxiv.org/html/2610.33764#A7.F13)shows the sample\-level source of this change: 89 previously incorrect decisions become valid and 6 previously valid decisions become invalid\. Explicit reasoning still leaves 101 of 195 decisions invalid, corresponding to 51\.79% of the paired cohort\.

The Thinking checkpoint reaches the 2,048\-token answer budget on all 195 answer\-only runs and the 256\-token finalization budget on all 195 two\-stage runs\. Its final JSON rate is 0\.00% for answer\-only decoding and 0\.51% after explicit reasoning\. The resulting valid\-answer rates of 0\.00% and 0\.51% characterize budget saturation and completion under this fixed protocol\. The Instruct comparison supplies the completed paired test of additional reasoning, while the Thinking checkpoint exposes generation budget as a separate systems constraint\.

#### Implication for the Integration Gap

The 42\.56\-point Instruct gain establishes that inference procedure contributes substantially to global validity\. The remaining 51\.79% error rate shows that a sizeable integration gap survives explicit step\-by\-step computation on the same visual evidence\. Reasoning budget thus joins visual presentation, model scale, and constraint structure as a variable that shapes multi\-evidence integration\. Future evaluations can separate reasoning quality from completion by scaling both deliberation and finalization budgets while preserving the paired local\-to\-global protocol\.

## Appendix HA Constructive Direction: Extract–Solve

The behavioral experiments reveal a persistent separation between recovering individual visual facts and combining those facts into a globally valid decision\. Appendix[G](https://arxiv.org/html/2610.33764#A7)shows that review, structured extraction, and explicit verification produce model\-dependent gains while difficult global decisions remain unresolved\. These findings motivate a direct division of labor: the vision\-language model converts the image into structured evidence, and a deterministic solver performs the joint constraint computation\.

#### Extract–Solve Pipeline

For each instance, the model first transcribes every card into a JSON record containing its item ID, cost, weight, value, color, type, and compatibility relations\. We reuse the transcription generated by the Extract–Select condition, giving 400 paired instances for every model and candidate count\. The resulting evaluation contains 4,800 transcriptions across four models and three values ofkk\.

A format audit identified several equivalent representations of compatibility relations\. Models expressed the same set as a JSON list, a comma\-separated string, a single item ID, or an empty\-set marker such asNoneornull\. The semantic parser maps these forms into a common set representation\. Records with missing cards, malformed JSON, unknown item IDs, invalid attributes, duplicate relations, or inconsistent compatibility fields receive a failed extraction outcome\. We report this semantic parsing analysis as an exploratory evaluation specified after the format audit\.

The deterministic component receives the structured transcription and a machine\-readable copy of the constraints displayed in the original prompt\. It enumerates all feasible subsets and emits an answer when the extracted instance yields one unique solution\. Zero\-solution and multiple\-solution cases receive separate outcome labels\. Gold metadata enters after prediction for scoring and for measuring exact transcription accuracy\.

#### Deterministic Integration Recovers Difficult Decisions

Table[A39](https://arxiv.org/html/2610.33764#A8.T39)compares Extract–Solve with the same\-runtime Direct and Extract–Select \(ES\) conditions defined in Appendix[G](https://arxiv.org/html/2610.33764#A7)\. The largest gains appear at the candidate counts where direct global decisions are weakest\. Qwen2\.5\-VL\-3B rises from 5\.75% to 92\.25% atk=8k=8, Qwen2\.5\-VL\-7B rises from 11\.75% to 96\.75%, and Gemma\-3\-12B rises from 14\.00% to 89\.50%\. Atk=6k=6, Qwen2\.5\-VL\-7B reaches 94\.50% and Gemma\-3\-12B reaches 100\.00%\. Their paired improvements over Direct range from 64\.75 to 86\.50 percentage points across these harder conditions\.

Table A39:Extract–Solve converts usable transcriptions into globally valid decisions\.All entries are percentages over 400 paired instances\. ES denotes Extract–Select\. Brackets beside Extract–Solve report pointwise Wilson 95% intervals\. The final column reports 95% intervals for the paired difference from Direct, estimated with 10,000 bootstrap draws stratified by conflict intensity and distractor ratio\.
#### Usable Transcriptions Lead to Correct Solutions

Figure[A14](https://arxiv.org/html/2610.33764#A8.F14)separates successful solutions from extraction and solver outcomes\. Across all models and candidate counts, 3,137 transcriptions yield one unique feasible solution\. Every one of these solutions matches the benchmark target, giving 3,137 correct predictions out of 3,137 solved instances\. The remaining outcomes comprise 1,553 incomplete card sets, 71 malformed or invalid records, and 39 transcriptions that produce zero or multiple feasible solutions\.

Figure A14:Outcome decomposition of the semantic Extract–Solve pipeline\.Each bar contains 400 instances\. Solved denotes transcriptions that yield one unique feasible solution; all 3,137 instances in this category match the benchmark target\. Incomplete denotes missing item cards, Invalid covers malformed JSON or invalid fields, and Inconsistent combines zero\- and multiple\-solution outcomes\. Q3 and Q7 denote Qwen2\.5\-VL\-3B and Qwen2\.5\-VL\-7B; G4 and G12 denote Gemma\-3\-4B and Gemma\-3\-12B\.
#### Extraction Completeness Determines End\-to\-End Performance

The model\-level differences are concentrated in transcription completeness\. Gemma\-3\-12B supplies a usable record for nearly everyk=4k=4andk=6k=6instance, supporting 99\.50% and 100\.00% end\-to\-end accuracy\. Qwen2\.5\-VL\-7B also maintains high completion rates and reaches 84\.00–96\.75% across candidate counts\.

Gemma\-3\-4B lists a subset of the available cards in 1,151 of its 1,200 instances, producing end\-to\-end accuracies between 0\.50% and 6\.50%\. Qwen2\.5\-VL\-3B shows a related completeness pattern atk=4k=4, where 252 of 400 transcriptions omit at least one card\. Every incomplete response ends before the 768\-token generation cap\. The model terminates its structured response while cards remain unlisted, identifying extraction completeness as an interface\-level failure\.

#### Decision\-Relevant Evidence

Exact transcription of every field sets a stricter requirement than successful selection\. For Gemma\-3\-12B atk=8k=8, 61\.50% of transcriptions match every evaluated field, while Extract–Solve reaches 89\.50% selection accuracy\. Some attribute and compatibility errors concern cards or relations outside the final feasible subset\. The unique valid decision remains recoverable when those errors preserve the constraints that determine the target\.

This pattern clarifies the role of extracted evidence\. Global success depends on preserving the facts that define the feasible solution, while peripheral transcription errors can leave the final decision unchanged\. The solver supplies an explicit test of whether the available transcription still supports a unique globally valid answer\.

#### From Diagnosis to a Constructive Design Principle

Extract–Solve turns the observed integration gap into a concrete system design\. A reliable visual\-to\-structured interface supplies the evidence, and deterministic computation enforces joint consistency across that evidence\. This combination nearly eliminates the difficultk=6k=6andk=8k=8failures for Qwen2\.5\-VL\-7B and Gemma\-3\-12B\. The Gemma\-3\-4B results identify extraction completeness as the next engineering target\.

These results establish a practical path for multimodal systems that operate over multiple pieces of evidence: visual models can specialize in faithful structured recovery, while explicit execution modules carry out globally constrained decisions\. This decomposition gives each component a clear responsibility, exposes its failures directly, and converts the local\-to\-global integration gap into an actionable system objective\.

## Appendix IDiscussion

#### The Missing Step Between Perception and Reasoning

Multimodal evaluation commonly treats the final answer as the primary unit of success, while compositional benchmarks reveal failures involving relations and combinations\([Yue et al\., 2024](https://arxiv.org/html/2610.33764#bib.bib1);[Wang et al\., 2025](https://arxiv.org/html/2610.33764#bib.bib2);[Liu et al\., 2024a](https://arxiv.org/html/2610.33764#bib.bib3);[Meng et al\., 2025](https://arxiv.org/html/2610.33764#bib.bib4);[Yuksekgonul et al\., 2022](https://arxiv.org/html/2610.33764#bib.bib7);[Ma et al\., 2023](https://arxiv.org/html/2610.33764#bib.bib8)\)\. Our results identify a more precise unit of analysis: the transformation from individually recoverable evidence into a decision that satisfies all constraints jointly\. This transformation deserves separate evaluation because local competence and global validity diverge sharply\. The gap survives same\-grid local questioning, appears across visual and textual inputs, persists in larger checkpoints, and extends from subset selection to weighted\-path reasoning\. These results locate a general challenge in coordinating available evidence during decision formation\.

This perspective changes how multimodal capability should be measured\. A model can identify every relevant object, recover its attributes, retrieve a specified target set, and still fail when several facts must jointly determine one answer\. Evaluation based on constituent recognition can assign high competence to systems whose decisions remain globally inconsistent\. Conditional local\-to\-global evaluation exposes this difference directly by asking how often demonstrated evidence recovery culminates in a valid decision\. It provides a diagnostic axis that complements broader measures of perception, knowledge, retrieval, and end\-to\-end reasoning\.

#### A Mechanistic Account Across Three Levels

The internal analyses connect three levels of evidence\. First, local queries establish that individual facts are behaviorally recoverable\. Second, membership probes show that global\-prompt hidden states contain information about which candidates belong to the valid solution\. Third, matched clean\-state replacement recovers failed decisions across substantial ranges of network depth\. The combination reveals a progression from available facts, to solution\-related representations, to decision sensitivity under intervention\.

This progression distinguishes our mechanism analysis from studies centered on probe accuracy or activation replacement alone\. Probe decodability measures information accessible to an external readout\. Matched and mismatched donors test whether the identity of the replacement state changes the generated decision\. Their agreement shows that solution\-related structure exists in the computation and that corresponding clean states can redirect corrupted decisions\. Their separation shows that accessible solution information may remain disconnected from the model’s final action\. The central mechanism is a conversion problem: internal states can carry useful evidence and solution membership while the generation process still produces a constraint\-violating subset\.

The depth and token\-scope results further characterize this conversion\. Early image\-token replacement produces strong recovery, while text\-position replacement becomes effective later in Qwen2\.5\-VL\-3B\. This profile is consistent with visual evidence becoming distributed across non\-image prompt positions through successive attention layers\. Full\-prompt replacement produces the strongest recovery near the later effective blocks, indicating that successful decisions depend on coordinated states across prompt positions\. The model\-specific recovery ranges add a second insight: architectures organize recoverable decision states differently\. InternVL retains matched\-donor effects deeper than Gemma, while larger Qwen and Gemma checkpoints change their recovery profiles in different directions\. Parameter count improves some behavioral outcomes, while architecture and training shape how long corrective information remains usable\.

#### From Mechanistic Diagnosis to System Design

The exploratory Extract–Solve analysis converts the diagnosis into a concrete design principle\. The vision\-language model transcribes the visual evidence, and a deterministic component executes the joint constraints\. Across all evaluated cohorts, every transcription that yields a unique feasible solution produces the correct benchmark answer, covering 3,137 out of 3,137 solved instances\. For Qwen2\.5\-VL\-7B and Gemma\-3\-12B, this pipeline raises difficultk=6k=6andk=8k=8conditions to 89\.50–100\.00% accuracy\. The result demonstrates that explicit execution can transform available structured evidence into reliable global decisions\. The remaining errors reveal the next systems challenge\. Gemma\-3\-4B frequently terminates after listing a subset of the cards, and Qwen2\.5\-VL\-3B shows a similar completeness pattern atk=4k=4\. The bottleneck has moved from joint constraint execution to the reliability of the visual\-to\-structured interface\. This decomposition creates two measurable objectives: faithful and complete evidence extraction, followed by verifiable global execution\. It also suggests architectures in which multimodal representations feed explicit planning, optimization, or symbolic modules through a structured intermediate state\.

#### Implications for Future Research

The findings motivate a research agenda organized around evidence integration\. Benchmarks can pair local queries with globally verifiable decisions and report conditional global validity alongside aggregate accuracy\. Training objectives can reward consistency across all relevant facts, preserve solution\-related information through generation, and penalize answers that violate jointly active constraints\. Architectural work can develop structured interfaces that expose evidence to execution modules while retaining uncertainty about ambiguous visual fields\. Interpretability studies can connect behavioral controls, probe targets, matched counterfactual donors, token scopes, and downstream decisions within one experimental chain\.

The broader lesson concerns what it means for a multimodal model to reason successfully\. Recognition supplies the pieces of a decision; integration determines whether those pieces jointly support the action the model takes\. Our study makes this transition observable at the behavioral, representational, causal, and system levels\. Treating evidence integration as a first\-class capability opens a path toward models whose final decisions remain consistent with the information they already possess\.

## Appendix JExtended Related Work

#### Compositional Generalization in Vision–Language Models

Recent work directly tests whether vision–language models can compose skills that are learned or evaluated separately\. Li et al\. train models on isolated unimodal or reasoning capabilities and evaluate out\-of\-distribution cross\-modal and cross\-task combinations\([Li et al\., 2025](https://arxiv.org/html/2610.33764#bib.bib23)\)\. Their results show a compositional ability gap between individual skills and their joint use, and identify explicit visual description and progressive grounding as useful training signals\. Related studies evaluate sensitivity to object–attribute binding, relations, spatial configurations, and controlled recombination\([Yuksekgonul et al\., 2022](https://arxiv.org/html/2610.33764#bib.bib7);[Ma et al\., 2023](https://arxiv.org/html/2610.33764#bib.bib8);[Kamath et al\., 2023](https://arxiv.org/html/2610.33764#bib.bib9)\)\.

Our study examines composition within one exactly verifiable decision\. Each instance contains several visual facts whose numerical, categorical, and relational constraints jointly determine a valid subset\. Conditional same\-grid evaluation measures global validity after the same model has successfully recovered every constituent fact\. This setting connects local evidence recovery, exact constraint satisfaction, candidate\-level solution information, clean\-state replacement, and explicit execution without a training phase between the constituent and composed tasks\.

#### Evidence Use in Long and Multimodal Contexts

Long\-context evaluations reveal sensitivity to evidence position and difficulty retrieving or combining distributed information\([Liu et al\., 2024b](https://arxiv.org/html/2610.33764#bib.bib10);[Bai et al\., 2024](https://arxiv.org/html/2610.33764#bib.bib11);[Wang et al\., 2024a](https://arxiv.org/html/2610.33764#bib.bib13)\)\. NoLiMa targets retrieval beyond literal matching\([Modarressi et al\., 2025](https://arxiv.org/html/2610.33764#bib.bib12)\), while MileBench studies long\-context multimodal understanding\([Song et al\., 2024](https://arxiv.org/html/2610.33764#bib.bib22)\)\. Multi\-image benchmarks similarly measure comparison, relational reasoning, and reasoning across image collections and sequences\([Wang et al\., 2025](https://arxiv.org/html/2610.33764#bib.bib2);[Liu et al\., 2024a](https://arxiv.org/html/2610.33764#bib.bib3);[Meng et al\., 2025](https://arxiv.org/html/2610.33764#bib.bib4);[Wang et al\., 2024b](https://arxiv.org/html/2610.33764#bib.bib5)\)\. Our matched\-retrieval control tests whether the target cards can be located under the same visual presentation, and the conditional analysis asks whether successful evidence recovery culminates in a jointly valid decision\.

#### Probing and Causal Intervention

Probes test which variables are accessible from hidden representations, while activation patching tests whether replacing internal states changes model behavior\. Prior studies apply these tools to factual recall, latent reasoning, and visual information processing\([Geva et al\., 2023](https://arxiv.org/html/2610.33764#bib.bib21);[Yang et al\., 2024](https://arxiv.org/html/2610.33764#bib.bib16);[Neo et al\., 2025](https://arxiv.org/html/2610.33764#bib.bib17)\)\. Work on patching methodology also highlights the importance of donor construction, intervention scope, and controls for learned subspaces\([Zhang and Nanda, 2024](https://arxiv.org/html/2610.33764#bib.bib19);[Makelov et al\., 2024](https://arxiv.org/html/2610.33764#bib.bib20)\)\. Our analysis combines candidate\-level membership probes with matched and mismatched donors, depth comparisons, and token\-scope interventions\. The supplementary generator\-group and construction\-baseline controls test whether membership decoding extends beyond fixed candidate frequencies and benchmark\-generation features\.