Inside VLM Chart Reading: Tracing Value Reading from Vertical Bar Charts Across Space and Depth

# Inside VLM Chart Reading: Tracing Value Reading from Vertical Bar Charts Across Space and Depth
Source: [https://arxiv.org/html/2609.13745](https://arxiv.org/html/2609.13745)
Qingfu ZhuWanxiang CheAffiliation:Research Center for Social Computing and Interactive RoboticsAffiliation:Harbin Institute of Technology, China

###### Abstract

Vision–language models \(VLMs\) can answer chart questions accurately, but output accuracy does not show how they combine the evidence needed to recover an exact value\. We study vertical\-bar value reading with controlled counterfactual activation patching in Qwen2\.5\-VL\-7B\-Instruct and InternVL3\.5\-8B\. The study connects three analyses: \(1\) The single\-factor results show that the changed bar\-top region restores much more answer preference than the unchanged bar body, despite containing fewer visual tokens\. Legend\- and series\-related states also lose local recoverability earlier than bar\-geometry and axis\-scale states\. \(2\) In the handoff analysis, restoration shifts from visual legend regions in early layers to prompt\-series positions in middle layers\. Resetting the prompt\-series state selectively reduces legend\-source rescue, supporting its role as a partial mediator\. \(3\) In the factorial analysis, both models can use geometry and scale states from separate donors to favor the combined target\. InternVL performs similarly when the states come from separate donors or one image, while Qwen shows lower restoration for separate donors which suggests higher context sensitivity\. Together, these results provide preliminary causal evidence for localizing the internal computations that support exact bar\-value reading\.

## 1Introduction

Charts express numerical information through text, color, position, and geometric extent\. Recent vision–language models \(VLMs\)\([Bai et al\., 2025](https://arxiv.org/html/2609.13745#bib.bib16);[Wang et al\., 2025](https://arxiv.org/html/2609.13745#bib.bib15);[Team et al\., 2026](https://arxiv.org/html/2609.13745#bib.bib17)\)perform well on chart question answering benchmarks\([Masry et al\., 2022](https://arxiv.org/html/2609.13745#bib.bib1);[Xu et al\., 2024](https://arxiv.org/html/2609.13745#bib.bib6);[Wang et al\., 2024](https://arxiv.org/html/2609.13745#bib.bib5)\)\. However, but output accuracy alone does not explain how they recover exact values\.

To this end, we focus on reading the exact integer value of a specified vertical bar\. The prompt identifies a series and category; the legend and x\-axis labels connect them to a bar; the bar top gives a visual magnitude; and the y\-axis maps that magnitude to a number\. We therefore ask three questions: where each factor can affect the answer, whether legend information is handed to a language\-side state, and whether geometry and scale states from different inputs can be used together\.

We use controlled counterfactual activation patching to answer these questions\. Paired charts or prompts differ in one or two known factor, and selected activation states are transferred between their forward passes\. We score complete answer sequences and measure normalized restoration of the relevant answer preference\. We evaluate Qwen2\.5\-VL\-7B\-Instruct and InternVL3\.5\-8B on matched examples\.

Thesingle\-factor analysisseparately changes axis scale, bar height, legend mapping, x\-axis labels, and prompt targets\. For bar height, we compare the full affected bar, the changed region around its top, and the unchanged body using the same chart pairs\. Across both models, the changed top restores substantially more answer preference than the unchanged body and recovers most of the full\-bar effect\. This result remains after controlling for token count\. Across depth, legend\- and series\-related states lose their local effect earlier than bar\-geometry and axis\-scale states\. Spatial expansion changes restoration magnitude more than the overall depth pattern\.

Thelegend\-to\-language handoff analysisfollows legend information after it becomes weak at its original image location\. Local restoration is strongest at legend regions in early layers, prompt\-series positions in middle layers, and the final prompt position in later layers\. Because this ordering alone does not show information transfer, we add a held\-out reset\-based blocking test\. Resetting the prompt\-series state removes part of the effect produced by restoring the legend, while matched language and visual controls have little effect\. This supports a partial mediating role for the prompt\-series state\.

Thefactorial recombination analysistests whether bar geometry and axis scale can be combined across forward passes\. In each2×22\\times 2family, a geometry donor changes only target\-bar height, a scale donor changes only axis scale, and a diagonal chart contains both changes\. Joint\-cross patches geometry and scale states from the two separate donors into one receiver computation; joint\-coherent takes both states from the diagonal\. Both models usually favor the combined target under joint\-cross\. InternVL changes little between the two joint conditions, whereas Qwen gives lower restoration when the states come from separate donors\. Thus, downstream computation can use separately obtained geometry and scale states together\.

Our contributions are threefold:

- •Single\-factor localization\.We separate the local effects of target grounding, bar height, axis scale, and prompt information, showing stronger value\-specific restoration at the changed bar top and earlier loss of legend\- and series\-related states\.
- •Legend\-to\-language handoff\.A held\-out reset test shows that the prompt\-series state carries part of the effect produced by restoring the visual legend\.
- •Cross\-factor combination\.A factorial intervention shows that both models can jointly use bar\-geometry and axis\-scale states taken from separate forward passes, while differing in sensitivity to donor context\.

## 2Related work

##### Chart understanding and diagnostic evaluation\.

Chart question answering has progressed from controlled benchmarks such as FigureQA\([Kahou et al\., 2018](https://arxiv.org/html/2609.13745#bib.bib2)\), PlotQA\([Methani et al\., 2020](https://arxiv.org/html/2609.13745#bib.bib3)\), and ChartQA\([Masry et al\., 2022](https://arxiv.org/html/2609.13745#bib.bib1)\)to more diverse evaluations including ChartQAPro\([Masry et al\., 2025](https://arxiv.org/html/2609.13745#bib.bib4)\), CharXiv\([Wang et al\., 2024](https://arxiv.org/html/2609.13745#bib.bib5)\), ChartBench\([Xu et al\., 2024](https://arxiv.org/html/2609.13745#bib.bib6)\), and EncQA\([Mukherjee et al\., 2026](https://arxiv.org/html/2609.13745#bib.bib7)\), with broader studies further documenting the limitations of general\-purpose VLMs on chart reasoning\. Chart\-specialized models such as DePlot\([Liu et al\., 2023a](https://arxiv.org/html/2609.13745#bib.bib18)\), MatCha\([Liu et al\., 2023b](https://arxiv.org/html/2609.13745#bib.bib20)\), and UniChart\([Masry et al\., 2023](https://arxiv.org/html/2609.13745#bib.bib19)\)improve visual extraction and numerical reasoning, but output\-level evaluation alone cannot determine how model handles target grounding, geometric measurement, scale interpretation internally\.

Most closely related, FUGU\([Tartaglini et al\., 2025](https://arxiv.org/html/2609.13745#bib.bib8)\)uses synthetic scatter plots, vision\-encoder activation patching, and linear probes to diagnose coordinate\-extraction and vision–language handoff bottlenecks\. In contrast, we study successful exact value reading in vertical bar charts using component\-wise image and prompt counterfactuals with geometry\-aligned patching across language\-model depth, separating bar\-endpoint magnitude, axis\-scale, and target\-binding information\.

##### Activation patching and multimodal interpretability\.

Causal tracing and activation patching replace hidden states between paired inputs and measure the resulting change in model behavior\([Meng et al\., 2023](https://arxiv.org/html/2609.13745#bib.bib9)\)\. These interventions support circuit localization and comparisons of information availability across depth\([Conmy et al\., 2023](https://arxiv.org/html/2609.13745#bib.bib10)\)\. However, conclusions can depend strongly on how the corrupted input is constructed, which output metric is used, and whether interventions target neurons, tokens, layers, or larger components\([Zhang and Nanda, 2024](https://arxiv.org/html/2609.13745#bib.bib11)\)\.

Mechanistic analysis has increasingly been extended to multimodal models\. Early work adapts causal tracing to BLIP for image\-conditioned generation\([Palit et al\., 2023](https://arxiv.org/html/2609.13745#bib.bib13)\)\. NOTICE replaces unstructured Gaussian corruption with semantic image perturbations and aligned text\-token replacements, enabling more interpretable causal mediation experiments in VLMs\([Golovanevsky et al\., 2025](https://arxiv.org/html/2609.13745#bib.bib14)\)\. Recent work also patches broad sets of image, query, and output\-token states to distinguish direct from text\-mediated routes of visual information\([Salazar et al\., 2026](https://arxiv.org/html/2609.13745#bib.bib12)\)\.

We build on these principles with chart\-program\-derived counterfactuals and geometry\-derived ROIs aligned to each model’s visual token grid\. Original states are inserted into a counterfactual computation whose correct numerical answer has changed, and restoration is measured using the complete answer sequence rather than a single output token\. We further compare strict any\-overlap mapping with merge\-closed spatial neighborhoods and repeat interventions over individual layers and contiguous layer windows\.

## 3Preliminary

### 3\.1Task and Controlled Charts Value Reading Data Generation

##### Task Definition

Each example contains a vertical bar chart and a query of the form:*Read the exact y\-axis value of the “SERIES” bar at “LABEL”\. Return only an integer\.*

##### Chart Value Reading Data Generation

Series and category names are semantically meaningless pseuwords\. Charts contain 1–3 series and 4–6 categories; the y\-axis begins at zero, with major tick steps in\{5,10\}\\\{5,10\\\}\. y\-max≤\\leq60\. Layout slots are fixed and grids, minor ticks, hatching, and decorative elements are removed\. The same chart semantics are rendered at1344×13441344\{\\times\}1344for Qwen2\.5\-VL and448×448448\{\\times\}448for InternVL3\.5\. The code template for each synthesized chart is shown in Figure[9](https://arxiv.org/html/2609.13745#A1.F9)and related parameters are randomly sampled\. We synthesize 500 charts in total\. For each chart and each bar in chart, we construct a value reading sample using the template in task definition and this result in 5932 samples\. Detailed distribution are shown in appendix[A\.1](https://arxiv.org/html/2609.13745#A1.SS1)\.

### 3\.2Counterfactual Chart Generation

ModalityCounterfactualCounterfactual generationPatched support / ROI definitionImageAxis scaleMultiply all y\-axis tick values and the corresponding target value by22, while preserving the rendered bar geometry\. In the formal patching experiments, the original and counterfactual members are randomly exchanged for approximately half of the pairs\.The y\-axis scale region, including the affected tick labels and their associated scale support\. Example is shown in Figure[19](https://arxiv.org/html/2609.13745#A3.F19)\.Bar heightRandomly increase or decrease the target bar height by exactly one y\-axis tick step, subject to the valid plotting range\.Union:union of the target\-bar regions in the original and counterfactual images\. Example is shown in Figure[15](https://arxiv.org/html/2609.13745#A3.F15)Difference:symmetric difference between the two target\-bar regions, concentrated around the changed bar top\. Example is shown in Figure[14](https://arxiv.org/html/2609.13745#A3.F14)Intersection:intersection of the two target\-bar regions, corresponding to the invariant shared bar body\. Example is shown in Figure[16](https://arxiv.org/html/2609.13745#A3.F16)Legend swapRandomly exchange the legend mapping of the target bar’s series with that of another eligible series\.The two affected legend\-entry regions, including their visual keys and associated text labels\. Example is shown in Figure[17](https://arxiv.org/html/2609.13745#A3.F17)X\-label swapRandomly exchange the target bar’s category label with another eligible category label\.The two affected x\-axis label slots\. Example is shown in Figure[18](https://arxiv.org/html/2609.13745#A3.F18)PromptLabelKeep the image fixed and replace the target category expression with another valid category, while preserving token positions and token count required for activation patching\.The prompt\-token positions corresponding to the category expression\.SeriesKeep the image fixed and replace the target series expression with another valid series, with position\-aligned prompt tokens\.The prompt\-token positions corresponding to the series expression\.Label \+ seriesKeep the image fixed and replace both the target category and target series expressions with position\-aligned alternatives\.The union of the prompt\-token positions corresponding to the category and series expressions\.Table 1:Counterfactual construction and patched supports for the nine analysis conditions\. The three bar conditions share the same one\-tick bar\-height counterfactual and differ only in the spatial ROI used for activation patching\.For each of the 5,932 query\-level samples, we apply the corresponding image\-editing strategy in Table[1](https://arxiv.org/html/2609.13745#S3.T1)to its associated chart, yielding an initial set of 5,932 original–counterfactual pairs for each image counterfactual type\.

### 3\.3Evaluation Results on Original and Counterfactal Chart Value Reading

Before analyzing internal activations, we verify that the image counterfactuals in Table[1](https://arxiv.org/html/2609.13745#S3.T1)have the intended behavioral effects\. The 500 source charts yield 5,932 value\-reading queries, and each counterfactual is evaluated using the same prompt on the original and edited charts\. A*target\-relevant*edit changes visual evidence that determines the queried value\. A*non\-target*edit instead applies an analogous operation away from the queried bar: changing another bar, swapping two category labels that exclude the target category, or swapping two legend mappings that exclude the target series\. The gold answer therefore remains unchanged under a non\-target edit\. We do not construct a non\-target axis\-scale condition because the axis scale globally affects the numerical interpretation of all bars\.

On the original charts, strict accuracy is97\.57%97\.57\\%for Qwen2\.5\-VL\-7B and97\.27%97\.27\\%for InternVL3\.5\-8B\. For target\-relevant bar\-height, legend\-swap, and x\-label\-swap counterfactuals, both endpoints are answered correctly for95\.03%95\.03\\%–95\.52%95\.52\\%of the pairs\. Axis scaling is more difficult, with both\-endpoint correctness of72\.02%72\.02\\%for Qwen2\.5\-VL\-7B and84\.95%84\.95\\%for InternVL3\.5\-8B\. For non\-target edits, the strict\-correctness status is preserved on99\.36%99\.36\\%–99\.90%99\.90\\%of the pairs, indicating that local edits unrelated to the queried bar rarely change whether the answer is correct\. The full results are shown in appendix[A\.2](https://arxiv.org/html/2609.13745#A1.SS2)\.

## 4Activation Patching Framework

### 4\.1Activation patching and metric

We score complete answer token sequences with teacher forcing\. This avoids reducing multi\-token numbers to their first token\. For an inputxx, queryqq, and answer sequencea=\(a1,…,aT\)a=\(a\_\{1\},\\ldots,a\_\{T\}\),

Sx​\(a\)=∑t=1Tlog⁡P⁡\(at∣x,q,a<t\)\.S\_\{x\}\(a\)=\\sum\_\{t=1\}^\{T\}\\log P\(a\_\{t\}\\mid x,q,a\_\{<t\}\)\.\(1\)Letaoa\_\{o\}andaca\_\{c\}be the original and counterfactual gold answers\. Define the original, counterfactual, and patched margins:

Mo\\displaystyle M\_\{o\}=So​\(ao\)−So​\(ac\),\\displaystyle=S\_\{o\}\(a\_\{o\}\)\-S\_\{o\}\(a\_\{c\}\),\(2\)Mc\\displaystyle M\_\{c\}=Sc​\(ao\)−Sc​\(ac\),\\displaystyle=S\_\{c\}\(a\_\{o\}\)\-S\_\{c\}\(a\_\{c\}\),\(3\)Mp\\displaystyle M\_\{p\}=Sp​\(ao\)−Sp​\(ac\)\.\\displaystyle=S\_\{p\}\(a\_\{o\}\)\-S\_\{p\}\(a\_\{c\}\)\.\(4\)We report

R=Mp−McMo−Mc\.R=\\frac\{M\_\{p\}\-M\_\{c\}\}\{M\_\{o\}\-M\_\{c\}\}\.\(5\)R=1R=1restores the full endpoint margin gap andR=0R=0leaves the counterfactual preference unchanged\. We do not clipRR\. No evaluated sample has\|Mo−Mc\|<10−6\|M\_\{o\}\-M\_\{c\}\|<10^\{\-6\}\.

All patches are original→\\rightarrowcounterfactual\. Image conditions patch the projected image tokens, individual post\-residual language layers, and contiguous windows of two or four layers\. Prompt conditions patch changed prompt positions at individual layers and windows\. We interpret restoration as*causal recoverability*: an original local state is sufficient to shift the counterfactual computation toward the original answer\. It does not establish that the region is the only necessary evidence\.

We use normalized restorationRRas the outcome measure for all interventions \(Eq\.[16](https://arxiv.org/html/2609.13745#A2.E16)\)\. For image interventions,*projected\-site restoration*patches the ROI\-selected visual tokens at the projected image\-token site \(Eq\.[18](https://arxiv.org/html/2609.13745#A2.E18)\), whereas*layer\-wise restoration*applies the same patch at individual language layers \(Eq\.[19](https://arxiv.org/html/2609.13745#A2.E19)\)\. Prompt interventions patch the changed category or series token positions at each language layer and therefore have no projected\-site image condition \(Eq\.[20](https://arxiv.org/html/2609.13745#A2.E20)\)\.

### 4\.2Models and spatial mappings

We evaluate Qwen2\.5\-VL\-7B\-Instruct\([Bai et al\., 2025](https://arxiv.org/html/2609.13745#bib.bib16)\)and InternVL3\.5\-8B\([Wang et al\., 2025](https://arxiv.org/html/2609.13745#bib.bib15)\)\. Qwen2\.5\-VL produces a96×9696\{\\times\}96vision grid, which is merged into a48×4848\{\\times\}48image\-token grid before its 28 language layers\. InternVL3\.5 similarly uses a32×3232\{\\times\}32vision grid, a16×1616\{\\times\}16merged image\-token grid, and 36 language layers\. We do not patch the fine vision grids; all image interventions operate on merged image\-token positions\.

For each image ROI in Table[1](https://arxiv.org/html/2609.13745#S3.T1), theany\-overlaprule selects every merged image token whose spatial cell overlaps at least one ROI pixel\. This gives a set of positions on the48×4848\{\\times\}48grid for Qwen2\.5\-VL and on the16×1616\{\\times\}16grid for InternVL3\.5\.

For Qwen2\.5\-VL, we additionally evaluate a3×33\{\\times\}3coarsemapping\. Each token selected by any\-overlap is expanded to its containing non\-overlapping3×33\{\\times\}3block on the merged grid\. This includes nearby context and reduces sensitivity to token boundaries; it is used as a robustness condition rather than as an estimate of a minimal causal region\. InternVL3\.5 uses only any\-overlap\.

The selected merged\-token positions are shared across patching sites\. For*projected\-site restoration*, they are patched once at the projected image\-token representation entering the language model\. For*layer\-wise restoration*, the same positions are patched separately at each language layer\. Thus, the two analyses differ in where the states are replaced, not in which spatial tokens represent the ROI\. Prompt interventions do not use these spatial mappings and patch only the designated prompt\-token positions\.

SettingAxisBar unionBar diff\.Bar inter\.LegendX\-labelQwen any\.643\.910\.783\.328\.968\.717Qwen3×33\{\\times\}3\.819\.950\.912\.420\.981\.784InternVL\.965\.994\.936\.490\.9181\.000Table 2:Mean projected\-site normalized restoration for all image interventions \(n=500n=500each\)\. Prompt interventions do not have a projected\-image condition\.

## 5Factor\-Wise Casual Localization

In this section we mainly analysis the single\-factor defined in Table[8](https://arxiv.org/html/2609.13745#A1.T8)

### 5\.1Experiment Setup

##### Evaluation Data Auditing

The 500 source charts yield 5,932 series–category value\-reading queries\. We audit how these queries are filtered and whether the final evaluation data cover varied chart structures and intervention directions\. The final evaluation contains 500 queries for each of the nine conditions\.

Here, a*bundle*means that several conditions reuse the same 500 original query sources, allowing paired comparisons\. The visual bundle reuses the same target bars for the three bar ROIs, legend swap, and x\-label swap; in particular, bar union, difference, and intersection use the exact same original–counterfactual image pairs and differ only in the patched ROI\. The prompt bundle reuses another 500 original image–query sources for the label, series, and label\+series prompt edits\. Axis scale uses a separate query pool\. Within every condition, the two models use identical sample IDs, order, and intervention metadata\. The three pools together cover 464 source charts and 1,361 distinct target bars, with near\-balanced intervention directions and no missing bundles\. The auditing result suggest that there may not be obvious bias in the final evaluation result\. More details are provided in Appendix[C](https://arxiv.org/html/2609.13745#A3)\.

##### Related Evaluation Statistic Metrics

The normalized restoration R are used to compute the statistic metrics\. We report the query\-weighted mean and a 5,000\-resample bootstrap 95% confidence interval \(Eqs\.[21](https://arxiv.org/html/2609.13745#A2.E21)–[24](https://arxiv.org/html/2609.13745#A2.E24)\)\. Conditions evaluated on the same queries are compared using within\-query paired differences \(Eqs\.[25](https://arxiv.org/html/2609.13745#A2.E25)–[26](https://arxiv.org/html/2609.13745#A2.E26)\)\. We additionally use source\-chart cluster bootstrap and chart\-equal estimates to account for multiple queries derived from the same chart \(Eqs\.[29](https://arxiv.org/html/2609.13745#A2.E29)and[31](https://arxiv.org/html/2609.13745#A2.E31)\)\. The main bar\-region comparison is also repeated on queries for which the changed\-top ROI selects no more tokens than the invariant\-body ROI \(Eqs\.[34](https://arxiv.org/html/2609.13745#A2.E34)–[35](https://arxiv.org/html/2609.13745#A2.E35)\)\.

For layer\-wise analyses, layer depth is normalized asdℓ=ℓ/\(L−1\)d\_\{\\ell\}=\\ell/\(L\-1\)\(Eq\.[38](https://arxiv.org/html/2609.13745#A2.E38)\)\. Raw AUC summarizes total restoration across depth, while retention AUC summarizes persistence after normalizing each profile by its layer\-0 value \(Eqs\.[39](https://arxiv.org/html/2609.13745#A2.E39)and[43](https://arxiv.org/html/2609.13745#A2.E43)\)\. Relative half\-life is the first normalized native\-layer depth at which the mean retention profile reaches0\.50\.5\(Eqs\.[45](https://arxiv.org/html/2609.13745#A2.E45)–[46](https://arxiv.org/html/2609.13745#A2.E46)\)\. Two\- and four\-layer windows test whether restoration is distributed across neighboring layers \(Eq\.[48](https://arxiv.org/html/2609.13745#A2.E48)\)\. Spatial\-mapping and cross\-model profile correlations are defined in Sections[B\.16](https://arxiv.org/html/2609.13745#A2.SS16)and[B\.17](https://arxiv.org/html/2609.13745#A2.SS17)\. Full metric definitions and computation details are provided in Appendix[B](https://arxiv.org/html/2609.13745#A2)\.

### 5\.2Projected\-site normalized restoration Analysis

#### 5\.2\.1Overall restoration across visual interventions

Table[2](https://arxiv.org/html/2609.13745#S4.T2)provides the complete projected\-site overview\. Whole\-bar, legend, and InternVL x\-label patches approach full restoration\. Axis and x\-label restoration vary more strongly with architecture and token mapping\. The bar\-region ordering, by contrast, is identical in all settings and motivates our primary paired analysis\.

#### 5\.2\.2The changed bar top dominates the invariant body

Figure[1](https://arxiv.org/html/2609.13745#S5.F1)reports projected\-site restoration\. The ordering is identical in all settings:RUnion\>RDifference≫RIntersectionR\_\{\\textsc\{Union\}\}\>R\_\{\\textsc\{Difference\}\}\\gg R\_\{\\textsc\{Intersection\}\}\. MeanDifferencerestoration is0\.7830\.783,0\.9120\.912, and0\.9360\.936for Qwen2\.5\-VL any\-overlap, Qwen2\.5\-VL coarse, and InternVL3\.5, compared with0\.3280\.328,0\.4200\.420, and0\.4900\.490forIntersection\. The pairedDifference–Intersectiongaps are0\.4550\.455\[0\.430, 0\.478\],0\.4920\.492\[0\.470, 0\.513\], and0\.4450\.445\[0\.420, 0\.471\]\. The direction holds for 94\.6%, 97\.6%, and 95\.6% of queries, respectively, and for 89\.6% of queries simultaneously across all three settings\.

Figure 1:Projected\-site normalized restoration for the three ROIs over the same bar\-height counterfactual\. Error bars are bootstrap 95% CIs\.The comparison isolates value\-specific visual evidence\. TheIntersectioncontains color, horizontal position, and most of the target bar’s body, but excludes the height change\. It still produces nonzero restoration, consistent with contextual or identity information\. However, the changed top is substantially more informative about the answer value\.

The result is not explained by patching a larger region\. Mean merged\-token counts forDifferenceversusIntersectionare 21\.8 vs\. 53\.2 \(Qwen2\.5\-VL any\), 47\.0 vs\. 97\.6 \(Qwen2\.5\-VL coarse\), and 5\.3 vs\. 11\.1 \(InternVL3\.5\)\. Restricting to examples whereDifferenceselects no more tokens thanIntersectionyields gaps of0\.4490\.449,0\.4800\.480, and0\.4380\.438\. TheDifferencealso captures most of the whole\-bar effect: its mean sample\-levelDifference/Unionratio is 0\.857, 0\.960, and 0\.941; median ratios are 0\.944, 0\.994, and 0\.993\.

Source\-chart dependence does not account for the effect\. Cluster\-bootstrap intervals forDifference–Intersectionare \[0\.431, 0\.478\], \[0\.471, 0\.513\], and \[0\.420, 0\.471\], nearly identical to query\-level intervals\. Chart\-equal effects are 0\.455, 0\.490, and 0\.441\. Thus the conclusion holds whether the estimand weights queries or charts equally\.

#### 5\.2\.3Spatial expansion changes magnitude

The Qwen2\.5\-VL coarse mapping increases projected restoration for all image interventions \(Figure[2](https://arxiv.org/html/2609.13745#S5.F2)\)\. Gains are largest for axis scale \(\+0\.175\), the changed bar top \(\+0\.129\), the invariant body \(\+0\.092\), and x\-labels \(\+0\.067\); legend gains only \+0\.013 because any\-overlap is already near saturation\. The effect is not a monotonic consequence of token expansion: legend tokens roughly double with little gain, whereas x\-label tokens expand 2\.61×\\timesbut gain less than axis scale\.

Figure 2:Paired projected\-restoration gain from Qwen2\.5\-VL3×33\{\\times\}3coarse neighborhoods over any\-overlap\.Figure 3:Mean single\-layer restoration for image\-level representative binding and quantitative interventions\. The absolute heights reflect both initial and later accessibility of each patch ROI\.Figure 4:Mean single\-layer restoration for prompt\-level representative interventions\.Figure 5:Layer profiles after normalization by each intervention’s layer\-0 restoration\. This view separates relative retention from the amount of information initially captured by an ROI\.

### 5\.3Layer\-wise Analysis

##### Legend and series\-related evidence declines earlier than bar geometry and axis scale

Figure[3](https://arxiv.org/html/2609.13745#S5.F3)compares single\-layer restoration over normalized depth\. Legend and series\-related information declines earlier than bar geometry and axis scale\. In InternVL3\.5, the mean profile falls sharply around layers 14–15 for series/legend, around layer 19 for category/x\-label, and around layers 23–24 for axis/bar information\. Qwen2\.5\-VL exhibits the same broad separation for legend versus quantitative evidence, although x\-label and prompt\-label timing is more architecture\-specific\.

Absolute threshold crossings can be misleading because a strict ROI may start with a lower restoration ceiling\. We therefore pair raw AUC with retention AUC\. Relative to legend, bar difference has a raw\-AUC advantage of 0\.058 \[0\.042, 0\.073\] for Qwen2\.5\-VL any, 0\.130 \[0\.118, 0\.143\] for Qwen2\.5\-VL coarse, and 0\.251 \[0\.238, 0\.264\] for InternVL3\.5\. The corresponding retention\-AUC advantages are 0\.169, 0\.174, and 0\.259, with all intervals excluding zero\. Hence bar geometry both carries more cumulative recoverable information and retains a larger fraction of its early effect deeper into the network\. Figure[5](https://arxiv.org/html/2609.13745#S5.F5)visualizes this amplitude\-normalized comparison\.

Axis scale remains especially late in Qwen2\.5\-VL: its relative half\-life occurs at normalized depth 0\.815 \(any\) and 0\.852 \(coarse\), versus approximately 0\.59–0\.63 for bar ROIs\. In InternVL3\.5, axis and bar half\-lives are closer \(0\.657 and 0\.686\)\. We therefore do not claim a universal strict sequence of measuring height and then applying scale\. The robust statement is that series/legend binding becomes locally unrecoverable earlier than bar geometry and axis\-scale evidence\.

##### Spatial expansion changes how much information is restored at the projected image\-token site, while largely preserving when that information remains locally recoverable across language\-model depth\.

Although the coarse mapping increases projected\-site restoration magnitude, it has little effect on the shape of the subsequent language\-layer restoration profiles\. For each intervention, we average the layer\-wise restorationRi,k,ℓmR\_\{i,k,\\ell\}^\{m\}over queries separately under the any\-overlap and3×33\{\\times\}3coarse mappings, and compute the within\-model profile correlationrkmapr\_\{k\}^\{\\mathrm\{map\}\}across Qwen’s native language layers \(Appendix[B\.16](https://arxiv.org/html/2609.13745#A2.SS16); Eqs\.[52](https://arxiv.org/html/2609.13745#A2.E52)–[54](https://arxiv.org/html/2609.13745#A2.E54)\)\. The mean correlation across all nine interventions is 0\.995, with correlations of 0\.998 for bar difference, 0\.9998 for bar intersection, and 0\.974 for axis scale\. The prompt profiles are identical because prompt\-token positions do not depend on the visual ROI mapping\.

##### The three bar ROIs have similar relative decay timing within a model despite different restoration magnitudes\.

The specific results are shown in Figure[3](https://arxiv.org/html/2609.13745#S5.F3)\. This suggests that the changed top and invariant body enter related downstream pathways, while differing mainly in the amount and specificity of information they carry\. \.

##### The prompt interventions show that category and series information exhibit distinct layer\-wise dynamics\.

The specific results are shown in Figure[4](https://arxiv.org/html/2609.13745#S5.F4)\. Prompt\-label interventions have a higher raw layer\-profile AUC than prompt\-series interventions, with paired mean advantages of 0\.058 \[0\.049, 0\.066\] in Qwen2\.5\-VL and 0\.113 \[0\.105, 0\.121\] in InternVL3\.5\. The paired difference is positive for 96\.2% and 97\.2% of queries, respectively\. Simultaneously replacing label and series yields an intermediate, non\-additive profile\. Category and series specifications therefore should not be treated as two independent linear channels\.

##### The two models show similar average layer\-wise restoration dynamics, but substantially weaker agreement in projected\-site restoration for individual queries\.

We first compare their language\-layer profiles\. For each interventionkk, we compute layer\-wise image\-token restorationRi,k,ℓmR\_\{i,k,\\ell\}^\{m\}at every language layer and average it over the 500 synchronized queries to obtain the native mean profileR¯k,ℓ\(m\)\\overline\{R\}\_\{k,\\ell\}^\{\(m\)\}\(Appendix[B\.17](https://arxiv.org/html/2609.13745#A2.SS17); Eqs\.[19](https://arxiv.org/html/2609.13745#A2.E19)and[55](https://arxiv.org/html/2609.13745#A2.E55)\)\. Because Qwen2\.5\-VL and InternVL3\.5 have different numbers of language layers, we map their native layers to normalized depth, linearly interpolate the two mean profiles onto the same 101\-point grid, and compute the profile correlationrkprofiler\_\{k\}^\{\\mathrm\{profile\}\}\(Eqs\.[56](https://arxiv.org/html/2609.13745#A2.E56)–[61](https://arxiv.org/html/2609.13745#A2.E61)\)\. The resulting correlations are 0\.951 for bar difference, 0\.945 for bar intersection, 0\.934 for legend, and 0\.799 for axis scale, with a mean of approximately 0\.898 across all nine interventions\. Thus, after averaging over queries, the two models tend to show similar changes in local recoverability across relative language\-layer depth\.

We separately compare per\-query projected\-site restorationRi,kproj,mR\_\{i,k\}^\{\\mathrm\{proj\},m\}, obtained by patching the ROI\-selected visual tokens at the projected image\-token site \(Eq\.[18](https://arxiv.org/html/2609.13745#A2.E18)\)\. This comparison does not use language\-layer profiles or depth interpolation: each query contributes one projected\-site restoration value from each model, and the two length\-500 vectors are correlated usingrkqueryr\_\{k\}^\{\\mathrm\{query\}\}\(Eq\.[63](https://arxiv.org/html/2609.13745#A2.E63)\)\. These correlations are much smaller: 0\.365 for bar difference, 0\.203 for bar intersection, and 0\.168 for axis scale\. The models therefore share a similar aggregate pattern over language\-layer depth, while often differing in which individual chart queries are strongly recoverable at the projected image\-token site\.

##### How Does Window Patching Affect Results

A computation may be distributed across several neighboring layers, so patching only one layer at a time could underestimate how long the relevant information remains accessible\. We therefore repeat the analysis by jointly patching contiguous windows of two or four layers\. As expected, window patching generally produces higher absolute restoration because more hidden states are replaced\. Importantly, however, it preserves the main ordering and transition regions observed in the single\-layer analysis: series and legend information is concentrated in earlier windows, whereas bar\-top and axis\-scale information remains recoverable in later windows\. Results are shown in Figure[12](https://arxiv.org/html/2609.13745#A3.F12)and Figure[13](https://arxiv.org/html/2609.13745#A3.F13)\.

## 6Legend\-to\-Language Handoff

The previous section showed that legend\-binding information stops being locally recoverable from the original legend location earlier than information associated with the other visual factors\. Here, we ask whether this information can be recovered at language\-side query positions after it has been recoded\. All statistical analyses in this section use normalized restoration,RR, as the outcome measure\.

### 6\.1Data

##### Evaluation data auditing\.

We reuse the 500 counterfactual queries from the*Legend swap*condition constructed in the previous section\. We split them by source chart into a pilot set of 100 queries and a held\-out set of 400 queries\. The two sets contain no overlapping source charts, preventing the same chart from contributing to both layer selection and the final effect estimates\.

### 6\.2Layer\-Wise Destination Analysis

#### 6\.2\.1Experimental Setup

We first examine how local causal recoverability at several candidate locations changes across model depth\. For each language\-model layer, we use the counterfactual input as the receiver computation and replace the activation at a candidate location with the corresponding activation from the original\-input forward pass\. Following the terminology of Table 1, the candidate locations are the two affected legend\-entry regions, including their visual keys and text labels, the target\-bar region, the prompt\-series token positions, and the final prompt token\. The target\-bar region is the union of the two bar regions at the queried category for the two series involved in the legend swap\. The prompt\-label token positions and a matched control\-bar region serve as controls\.

This analysis uses only the 400 held\-out queries\. The layer\-wise curves use 500 query\-level bootstrap resamples, and the shaded bands show pointwise 95% confidence intervals\.

#### 6\.2\.2Results

Figure 6:Layer\-wise normalized restoration at the main candidate locations for InternVL3\.5\-8B \(left\) and Qwen2\.5\-VL\-7B \(right\)\. Shaded bands denote pointwise 95% confidence intervals from 500 query\-level bootstrap resamples\.##### Single\-location activation patching reveals a consistent sequence of local causal recoverability in both models: early legend\-entry regions→\\rightarrowmiddle prompt\-series positions→\\rightarrowlate final\-prompt position\.

Figure[6](https://arxiv.org/html/2609.13745#S6.F6)shows the layer\-wise restoration profiles at the main candidate locations\. InternVL3\.5\-8B and Qwen2\.5\-VL\-7B exhibit the same broad depth ordering: restoration at the legend\-entry regions is concentrated in early layers, restoration at the prompt\-series positions is concentrated in middle layers, and restoration at the final prompt position is concentrated in late layers\. In contrast, the target\-bar region shows no restoration of substantive magnitude at any layer\.

Figure 7:Layer\-wise differences between each candidate destination and its matched control for InternVL3\.5\-8B \(left\) and Qwen2\.5\-VL\-7B \(right\)\. Shaded bands denote pointwise 95% confidence intervals from 500 query\-level bootstrap resamples\.
##### The local restoration effect is specific to the prompt\-series positions rather than the target\-bar region\.

Figure[7](https://arxiv.org/html/2609.13745#S6.F7)compares each candidate location with its matched control\. The difference between the target\-bar region and the matched control\-bar region remains close to zero, indicating that restoring the target\-bar activation alone does not consistently change the answer preference\. By contrast, the prompt\-series positions show stronger local restoration than the prompt\-label control positions, with the difference concentrated in the middle layers\.

The ordering of peaks across locations is not, by itself, causal evidence of information transfer or mediation\. Section[6\.3](https://arxiv.org/html/2609.13745#S6.SS3)therefore uses a reset\-based blocking intervention to test whether the prompt\-series positions carry part of the effect produced by restoring the legend source\.

### 6\.3Reset\-Based Mediation Analysis

#### 6\.3\.1Experimental Setup

We next test whether the prompt\-series positions carry part of the restoration effect produced at the legend source\.

##### Source rescue\.

Using the counterfactual input as the receiver computation, we replace the activations at the two affected legend\-entry regions with their original\-input activations at a selected source layer\. We refer to the resulting normalized restorationRRas*source rescue*\.

##### Layer selection on the pilot set\.

For each model, we first scan all layers on the 100 pilot queries and restore the original image\-token activations at the affected legend\-entry regions\. We select the layer with the largest mean source rescue as the source layer\. After fixing this source layer, we scan deeper candidate layers and reset the prompt\-series positions to their counterfactual activations\. The layer producing the largest decrease in source rescue is selected as the destination layer\. Both layers are then frozen and are not adjusted using the held\-out results\.

##### Effect estimation on the held\-out set\.

On the 400 held\-out queries, we first restore the original legend\-entry activations at the selected source layer to obtain the unblocked source\-rescue baseline\. We then reset the prompt\-series positions to their counterfactual activations at the selected destination layer and measure the resulting decrease in source rescue\. As controls, we separately reset the prompt\-label positions, the target\-bar region, and a matched control\-bar region at the same destination layer\. The control\-bar region is constructed at a randomly selected non\-query category\. These controls test whether the observed decrease is specific to the prompt\-series positions\. All blocking effects and their 95% confidence intervals are estimated on the held\-out set using 5,000 source\-chart cluster\-bootstrap resamples\.

#### 6\.3\.2Results

Resetting the prompt\-series state selectively blocks source rescue, supporting its role as a partial mediator of legend information\.For a reset locationdd, we define the blocking effect as

Δ​Rd=Rsource​only−Rsource\+reset⁡\(d\)\.\\Delta R\_\{d\}=R\_\{\\mathrm\{source\\ only\}\}\-R\_\{\\mathrm\{source\+reset\}\(d\)\}\.\(6\)A positiveΔ​Rd\\Delta R\_\{d\}indicates that resetting locationddremoves part of the source\-rescue effect\.

InternVL3\.5\-8BQwen2\.5\-VL\-7BReset typeReset locationPost\-resetRR\[95% cluster CI\]Blocking effectΔ​R\\Delta RPost\-resetRR\[95% cluster CI\]Blocking effectΔ​R\\Delta RNo\-reset baselineNone0\.910 \[0\.886, 0\.932\]–0\.967 \[0\.954, 0\.978\]–Prompt resetPrompt\-series positions0\.701 \[0\.676, 0\.727\]0\.2090\.541 \[0\.518, 0\.564\]0\.426Prompt resetPrompt\-label positions0\.909 \[0\.886, 0\.932\]0\.0010\.932 \[0\.918, 0\.944\]0\.035Visual resetTarget\-bar region0\.909 \[0\.886, 0\.932\]0\.0010\.967 \[0\.954, 0\.978\]0\.000Visual resetMatched control\-bar region0\.910 \[0\.886, 0\.932\]0\.0000\.967 \[0\.954, 0\.978\]0\.000

Table 3:Reset\-based blocking results on the 400 held\-out queries\. Post\-resetRRis normalized restoration after source rescue and the specified destination reset\. The blocking effect is the decrease relative to the no\-reset source\-rescue baseline\.As shown in Table[3](https://arxiv.org/html/2609.13745#S6.T3), resetting the prompt\-series positions substantially reduces source rescue in both models, whereas resetting the prompt\-label control positions produces a much smaller reduction\. Resetting either the target\-bar region or the matched control\-bar region has almost no effect on source rescue\. These results support the interpretation that the series identity specified by the legend is at least partly recoded into a language\-side representation at the prompt\-series positions, and that this representation contributes to the model’s answer preference\. This is intervention\-based evidence for a mediating role of the prompt\-series state; it does not establish that this state is the only pathway through which legend information affects the output\.

## 7Factorial Recombination of Bar Geometry and Axis Scale

### 7\.1Experimental Setup

This section tests whether information extracted from bar geometry and axis scale can be recombined across different model forward passes\.

Each family contains four charts formed by crossing two target\-bar heights with two axis\-scale settings\. One chart is used as the receiver\. Relative to the receiver, the*geometry donor*changes only the target\-bar height while keeping the axis scale fixed, and the*scale donor*changes only the axis scale while preserving the rendered target\-bar height\. The remaining chart, called the*diagonal*, contains both alternative factors and therefore represents the target combination of bar geometry and axis scale\.

We apply four interventions during the receiver forward pass while keeping the prompt unchanged:

- •Geometry\-onlyreplaces activations at the target\-bar region with those from the geometry donor\. This condition tests whether the transferred bar\-geometry information can affect the answer on its own\.
- •Scale\-onlyreplaces activations at the y\-axis scale region with those from the scale donor\. This condition tests whether the transferred axis\-scale information can affect the answer on its own\.
- •Joint\-crosstakes target\-bar\-region activations from the geometry donor and y\-axis\-scale\-region activations from the scale donor, and inserts both into the same receiver forward pass\. It tests whether information from two different donor forward passes can be combined to recover a target answer that neither donor provides by itself\.
- •Joint\-coherenttakes both sets of activations from the diagonal and inserts them together into the same receiver forward pass\. Comparing this condition with joint\-cross tests whether joint restoration depends on the two activation sets coming from the same image context\.

We perform these interventions both at the projected image\-token site and at individual language layers\.

### 7\.2Evaluation Metrics

For each family, the model scores the four candidate values as complete answer sequences\. For a candidate answerc=\(c1,…,cT\)c=\(c\_\{1\},\\ldots,c\_\{T\}\), its sequence score under conditionxxis

Sx​\(c\)=∑t=1Tlog⁡P⁡\(ct∣x,q,c<t\),S\_\{x\}\(c\)=\\sum\_\{t=1\}^\{T\}\\log P\\\!\\left\(c\_\{t\}\\mid x,q,c\_\{<t\}\\right\),\(7\)
whereqqis the fixed prompt andxxspecifies the input image and, when applicable, the intervention\. The score is the sum of the conditional log\-probabilities of all answer tokens\. The four candidates within a family have the same number of scored tokens, so we do not apply length normalization\.

#### 7\.2\.1Four\-Candidate Target Accuracy

At the projected image\-token site, we report four\-candidate target accuracy for all four interventions\. Each intervention has a predefined target answer: the correct answer of the geometry donor for geometry\-only, the correct answer of the scale donor for scale\-only, and the correct answer of the diagonal for both joint\-cross and joint\-coherent\.

A family is counted as a hit if the predefined target has a strictly higher sequence score than each of the other three candidates\. Ties and lower\-ranked targets are counted as misses\. Target accuracy is the mean hit rate across families\. This metric measures answer selection among the four predefined candidates; it is not free\-generation accuracy\.

#### 7\.2\.2Margins and Normalized Restoration

To match the endpoint notation used in the preceding analyses, we denote the receiver input by counterfactual \(cf\) and the diagonal input by original \(ori\)\. Both use the same prompt\. Letccfc\_\{\\mathrm\{cf\}\}be the correct answer for the receiver andcoric\_\{\\mathrm\{ori\}\}the correct answer for the diagonal\. The two unpatched baseline margins are

Mcf\\displaystyle M\_\{\\mathrm\{cf\}\}=Scf​\(cori\)−Scf​\(ccf\),\\displaystyle=S\_\{\\mathrm\{cf\}\}\(c\_\{\\mathrm\{ori\}\}\)\-S\_\{\\mathrm\{cf\}\}\(c\_\{\\mathrm\{cf\}\}\),\(8\)Mori\\displaystyle M\_\{\\mathrm\{ori\}\}=Sori​\(cori\)−Sori​\(ccf\)\.\\displaystyle=S\_\{\\mathrm\{ori\}\}\(c\_\{\\mathrm\{ori\}\}\)\-S\_\{\\mathrm\{ori\}\}\(c\_\{\\mathrm\{cf\}\}\)\.\(9\)
Thus,McfM\_\{\\mathrm\{cf\}\}measures the model’s preference forcoric\_\{\\mathrm\{ori\}\}overccfc\_\{\\mathrm\{cf\}\}when the receiver is the input, whereasMoriM\_\{\\mathrm\{ori\}\}measures the same preference when the diagonal is the input\.

We define a separate patched margin for each joint intervention:

Mpatchjoint​\-​cross\\displaystyle M\_\{\\mathrm\{patch\}\}^\{\\mathrm\{joint\\text\{\-\}cross\}\}=Spatchjoint​\-​cross​\(cori\)\\displaystyle=S\_\{\\mathrm\{patch\}\}^\{\\mathrm\{joint\\text\{\-\}cross\}\}\(c\_\{\\mathrm\{ori\}\}\)−Spatchjoint​\-​cross​\(ccf\),\\displaystyle\\quad\-S\_\{\\mathrm\{patch\}\}^\{\\mathrm\{joint\\text\{\-\}cross\}\}\(c\_\{\\mathrm\{cf\}\}\),\(10\)Mpatchjoint​\-​coherent\\displaystyle M\_\{\\mathrm\{patch\}\}^\{\\mathrm\{joint\\text\{\-\}coherent\}\}=Spatchjoint​\-​coherent​\(cori\)\\displaystyle=S\_\{\\mathrm\{patch\}\}^\{\\mathrm\{joint\\text\{\-\}coherent\}\}\(c\_\{\\mathrm\{ori\}\}\)−Spatchjoint​\-​coherent​\(ccf\)\.\\displaystyle\\quad\-S\_\{\\mathrm\{patch\}\}^\{\\mathrm\{joint\\text\{\-\}coherent\}\}\(c\_\{\\mathrm\{cf\}\}\)\.\(11\)
The first margin is obtained by patching the target\-bar region from the geometry donor and the y\-axis scale region from the scale donor into the receiver forward pass\. The second is obtained by patching both regions from the diagonal into the receiver forward pass\.

The corresponding normalized restoration scores are

Rjoint​\-​cross\\displaystyle R\_\{\\mathrm\{joint\\text\{\-\}cross\}\}=Mpatchjoint​\-​cross−McfMori−Mcf,\\displaystyle=\\frac\{M\_\{\\mathrm\{patch\}\}^\{\\mathrm\{joint\\text\{\-\}cross\}\}\-M\_\{\\mathrm\{cf\}\}\}\{M\_\{\\mathrm\{ori\}\}\-M\_\{\\mathrm\{cf\}\}\},\(12\)Rjoint​\-​coherent\\displaystyle R\_\{\\mathrm\{joint\\text\{\-\}coherent\}\}=Mpatchjoint​\-​coherent−McfMori−Mcf\.\\displaystyle=\\frac\{M\_\{\\mathrm\{patch\}\}^\{\\mathrm\{joint\\text\{\-\}coherent\}\}\-M\_\{\\mathrm\{cf\}\}\}\{M\_\{\\mathrm\{ori\}\}\-M\_\{\\mathrm\{cf\}\}\}\.\(13\)
For either joint condition,R=0R=0means that the intervention leaves the answer preference at the receiver \(cf\) baseline, whereasR=1R=1means that it restores the full preference difference between the receiver \(cf\) and diagonal \(ori\) baselines\.

#### 7\.2\.3Statistical Analysis Metrics

For the projected image\-token site, 95% confidence intervals for four\-candidate target accuracy,Rjoint​\-​crossR\_\{\\mathrm\{joint\\text\{\-\}cross\}\}, andRjoint​\-​coherentR\_\{\\mathrm\{joint\\text\{\-\}coherent\}\}are computed from 5,000 family\-level bootstrap resamples\.

The layer\-wise analysis reports separate curves forRjoint​\-​crossR\_\{\\mathrm\{joint\\text\{\-\}cross\}\}andRjoint​\-​coherentR\_\{\\mathrm\{joint\\text\{\-\}coherent\}\}; pointwise 95% confidence intervals use 500 family\-level bootstrap resamples\. In every analysis, the complete family is the sampling unit\.

### 7\.3Data Generation and Auditing

#### 7\.3\.1Initial Family Generation

We first generated 10,000 families and rendered each family at two resolutions:1344×13441344\\times 1344for Qwen2\.5\-VL\-7B and448×448448\\times 448for InternVL3\.5\-8B\. Apart from input resolution, the two model\-specific versions use exactly the same chart values, series names, category names, colors, layout, prompt, target position, and receiver–donor assignment\. Each family contains four charts that differ only in target\-bar geometry and axis scale\. Every chart contains two series and five categories, and the queried category is sampled from the three middle positions\.

Let the base tick interval bes∈\{5,10\}s\\in\\\{5,10\\\}and the initial target value bevv\. Crossing two target\-bar heights with two axis\-scale settings gives the four charts in Table[4](https://arxiv.org/html/2609.13745#S7.T4)\.

ChartBar geometryAxis scaleCorrect answerC00C\_\{00\}Base heightBase scalevvC10C\_\{10\}Alternate heightBase scalev\+sv\+sC01C\_\{01\}Base heightAlternate scale2​v2vC11C\_\{11\}Alternate heightAlternate scale2​\(v\+s\)2\(v\+s\)Table 4:The four charts in a factorial family\. The first index denotes bar\-geometry state and the second denotes axis\-scale state; 0 is the base state and 1 is the alternate state\.Changing bar geometry moves only the top of the target bar by one tick interval; all other bars and the axis scale remain unchanged\. Changing axis scale multiplies all bar values, the y\-axis tick interval, and the y\-axis upper limit by two\. This preserves the rendered geometry of every bar, so the visible change is concentrated in the y\-axis labels and their associated scale support\.

Receiver assignments are balanced across the four chart types and shuffled with a fixed random seed\. Relative to the assigned receiver, the geometry donor switches only the geometry state, the scale donor switches only the scale state, and the diagonal switches both\. Because every chart type can serve as the receiver, the evaluation includes both increases and decreases in bar height and axis scale\.

The generation pipeline also checks that the four correct answers are distinct\. It also confirms that the target\-bar and y\-axis scale regions do not overlap at either the pixel level or the model’s merged image\-token grids\.

#### 7\.3\.2Model Inference and Final Family Selection

We next asked Qwen2\.5\-VL\-7B and InternVL3\.5\-8B to answer all four unpatched charts in every family\. A response was counted as correct only if it could be parsed as a single integer and exactly matched the chart’s correct answer\.

Of the 10,000 generated families, Qwen2\.5\-VL\-7B answered all four charts correctly in 5,963 families, and InternVL3\.5\-8B did so in 7,264 families\. Both models answered all four charts correctly in 4,903 families\. We selected 500 of these cross\-model eligible families for activation patching, balancing both receiver type and base tick interval\. Selection used only unpatched answer correctness and did not use any activation\-patching result\.

#### 7\.3\.3Evaluation Data Distribution

Data dimensionDistribution in the final 500 familiesReceiver typeC00C\_\{00\},C10C\_\{10\},C01C\_\{01\}, andC11C\_\{11\}: 125 eachBase tick interval5 and 10: 250 eachBar\-height directionIncrease: 250; decrease: 250Axis\-scale directionIncrease: 250; decrease: 250Target series positionFirst series: 246; second series: 254Target category positionSecond category: 150; third: 153; fourth: 197Four candidate valuesPooled range: 11–90; median: 32\.0Within\-family candidate spanRange: 21–55; mean: 35\.752Table 5:Distribution of the 500 families used in the factorial recombination experiments\. Target categories were sampled only from the three middle positions of each five\-category chart\. “All candidate values \(pooled\)” combines the four candidate values from all families, giving 2,000 values in total\. “Within\-family candidate span” is the difference between the largest and smallest candidate values in each family; its minimum, maximum, and mean summarize the 500 resulting spans\.The final evaluation set distribution is shown in Table[5](https://arxiv.org/html/2609.13745#S7.T5)\. The final evaluation set is balanced across receiver types, base tick intervals, and the directions of both factor changes\. The two target\-series positions are nearly evenly represented, all three eligible target\-category positions are covered, and the pooled candidate range of 11–90\. The final data therefore do not collapse onto one receiver type, one change direction, one query position, or a narrow value range\.

This evaluation set is nevertheless success\-conditioned: every selected family had to be answered correctly by both models before intervention\. The conclusions therefore apply to this clean\-correct distribution and should not be directly extended to families on which either model fails without intervention\.

### 7\.4Projected\-Site Composition Analysis

We first apply all four interventions at the projected image\-token site\. Four\-candidate target accuracy indicates whether the predefined target answer has the highest sequence score\. For the two joint interventions,Rjoint​\-​crossR\_\{\\mathrm\{joint\\text\{\-\}cross\}\}andRjoint​\-​coherentR\_\{\\mathrm\{joint\\text\{\-\}coherent\}\}measure how much the intervention moves the answer preference from the receiver \(cf\) baseline toward the diagonal \(ori\) baseline\. The results are shown in Table[6](https://arxiv.org/html/2609.13745#S7.T6)\.

ModelInterventionFour\-candidate target accuracy \[95% CI\]Rjoint​\-​crossR\_\{\\mathrm\{joint\\text\{\-\}cross\}\}\[95% CI\]Rjoint​\-​coherentR\_\{\\mathrm\{joint\\text\{\-\}coherent\}\}\[95% CI\]InternVL3\.5\-8BGeometry\-only0\.996 \[0\.990, 1\.000\]––InternVL3\.5\-8BScale\-only1\.000 \[1\.000, 1\.000\]––InternVL3\.5\-8BJoint\-cross0\.996 \[0\.990, 1\.000\]0\.977 \[0\.973, 0\.980\]–InternVL3\.5\-8BJoint\-coherent0\.996 \[0\.990, 1\.000\]–0\.977 \[0\.974, 0\.981\]Qwen2\.5\-VL\-7BGeometry\-only0\.974 \[0\.960, 0\.986\]––Qwen2\.5\-VL\-7BScale\-only0\.994 \[0\.986, 1\.000\]––Qwen2\.5\-VL\-7BJoint\-cross0\.908 \[0\.882, 0\.932\]0\.803 \[0\.795, 0\.811\]–Qwen2\.5\-VL\-7BJoint\-coherent0\.996 \[0\.990, 1\.000\]–0\.867 \[0\.860, 0\.873\]

Table 6:Four\-candidate target accuracy for all four interventions at the projected image\-token site, together with the normalized restoration score for each joint intervention\.InternVL3\.5\-8B combines bar\-geometry and axis\-scale information equally well whether the two activation sets come from separate donors or from the same diagonal input\.Geometry\-only and scale\-only both achieve high target accuracy, showing that either activation set can independently shift the model toward the corresponding single\-factor donor answer\. Joint\-cross and joint\-coherent have the same target\-accuracy point estimate and nearly identical normalized restoration, indicating little sensitivity to whether the two activation sets share the same source context\.

Qwen2\.5\-VL\-7B can also combine bar\-geometry and axis\-scale information across forward passes, but it is more sensitive to whether the two activation sets come from the same input\.Geometry\-only and scale\-only again achieve high target accuracy\. However, joint\-cross has lower target accuracy and normalized restoration than joint\-coherent\. Thus, Qwen selects the diagonal target in most joint\-cross families, but combines the two activation sets less effectively when they come from separate geometry and scale donors than when both come from the diagonal\.

### 7\.5Layer\-Wise Composition Analysis

Figure 8:Layer\-wise normalized restoration for joint\-cross and joint\-coherent\. Shading shows pointwise 95% confidence intervals from 500 family\-level bootstrap resamples\.The layer\-wise curves show at which network depths the transferred activation states can still influence the model’s answer preference\.

For InternVL3\.5\-8B, both joint interventions are effective in early layers but become progressively weaker in later layers\.Joint\-cross and joint\-coherent produce strong and closely matched normalized restoration across the early and much of the middle network, indicating that the remaining downstream computation can use the two transferred activation sets in a similar way regardless of whether they come from separate donors or from the diagonal\. As the patch is moved to later layers, both curves decline at similar rates because less downstream computation remains available to integrate the transferred information and redirect the answer\.

For Qwen2\.5\-VL\-7B, early\-layer interventions are also the most effective, but joint\-cross loses its effect earlier than joint\-coherent\.The joint\-cross curve remains below the joint\-coherent curve through the main effective intervention window, showing that Qwen uses activation states from separate donors less reliably than two states taken from the same diagonal input\. At later layers, both interventions have less ability to change the final answer, with the reduction occurring earlier and more clearly for joint\-cross\.

Overall, early\-layer patching leaves enough downstream computation to jointly use the transferred bar\-geometry and axis\-scale information, whereas late\-layer patching provides less opportunity to alter the model’s answer\. The similar InternVL curves indicate limited sensitivity to donor context, while the earlier decline of Qwen’s joint\-cross curve indicates greater context sensitivity\.

## 8Conclusion

We used controlled counterfactual activation patching to study how two VLMs read exact values from vertical bar charts\. The single\-factor analysis shows that value\-specific information is more strongly recoverable from the changed bar\-top region than from the unchanged bar body, while legend\- and series\-related states lose their local effect earlier than bar geometry and axis scale\. The legend\-to\-language analysis further shows that resetting the prompt\-series state selectively reduces the effect of restoring the visual legend, supporting its role as a partial mediator\. Finally, the factorial analysis shows that both models can jointly use bar\-geometry and axis\-scale states taken from separate forward passes, although Qwen is more sensitive than InternVL to whether the states share the same source context\. Together, these results provide preliminary causal evidence for localizing and relating the internal states that support exact bar\-value reading\.

## Limitations

The study is conditional on strict four\-endpoint success: both models must answer original and counterfactual inputs correctly\. This yields a clean margin\-based intervention metric but does not characterize failure cases\. The charts are synthetic, use fixed layouts, and cover only vertical bars with integer values; natural charts introduce OCR noise, irregular styling, truncated axes, annotations, and broader semantic priors\.

## References

- Baiet al\.\(2025\)S\. Bai, K\. Chen, X\. Liu, J\. Wang, W\. Ge, S\. Song, K\. Dang, P\. Wang, S\. Wang, J\. Tang, H\. Zhong, Y\. Zhu, M\. Yang, Z\. Li, J\. Wan, P\. Wang, W\. Ding, Z\. Fu, Y\. Xu, J\. Ye, X\. Zhang, T\. Xie, Z\. Cheng, H\. Zhang, Z\. Yang, H\. Xu, and J\. LinQwen2\.5\-vl technical report\.External Links:2502\.13923,[Link](https://arxiv.org/abs/2502.13923)Cited by:[§1](https://arxiv.org/html/2609.13745#S1.p1.1),[§4\.2](https://arxiv.org/html/2609.13745#S4.SS2.p1.1)\.
- Conmyet al\.\(2023\)A\. Conmy, A\. Mavor\-Parker, A\. Lynch, S\. Heimersheim, and A\. Garriga\-AlonsoTowards automated circuit discovery for mechanistic interpretability\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 16318–16352\.External Links:[Document](https://dx.doi.org/10.52202/075280-0719),[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/34e1dbe95d34d7ebaf99b9bcaeb5b2be-Paper-Conference.pdf)Cited by:[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px2.p1.1)\.
- Golovanevskyet al\.\(2025\)M\. Golovanevsky, W\. Rudman, V\. Palit, R\. Singh, and C\. EickhoffWhat do vlms notice? a mechanistic interpretability pipeline for gaussian\-noise\-free text\-image corruption and evaluation\.External Links:2406\.16320,[Link](https://arxiv.org/abs/2406.16320)Cited by:[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px2.p2.1)\.
- Kahouet al\.\(2018\)S\. E\. Kahou, V\. Michalski, A\. Atkinson, A\. Kadar, A\. Trischler, and Y\. BengioFigureQA: an annotated figure dataset for visual reasoning\.External Links:1710\.07300,[Link](https://arxiv.org/abs/1710.07300)Cited by:[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2023a\)F\. Liu, J\. M\. Eisenschlos, F\. Piccinno, S\. Krichene, C\. Pang, K\. Lee, M\. Joshi, W\. Chen, N\. Collier, and Y\. AltunDePlot: one\-shot visual language reasoning by plot\-to\-table translation\.External Links:2212\.10505,[Link](https://arxiv.org/abs/2212.10505)Cited by:[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2023b\)F\. Liu, F\. Piccinno, S\. Krichene, C\. Pang, K\. Lee, M\. Joshi, Y\. Altun, N\. Collier, and J\. M\. EisenschlosMatCha: enhancing visual language pretraining with math reasoning and chart derendering\.External Links:2212\.09662,[Link](https://arxiv.org/abs/2212.09662)Cited by:[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px1.p1.1)\.
- Masryet al\.\(2025\)A\. Masry, M\. S\. Islam, M\. Ahmed, A\. Bajaj, F\. Kabir, A\. Kartha, M\. T\. R\. Laskar, M\. Rahman, S\. Rahman, M\. Shahmohammadi, M\. Thakkar, M\. R\. Parvez, E\. Hoque, and S\. JotyChartQAPro: a more diverse and challenging benchmark for chart question answering\.External Links:2504\.05506,[Link](https://arxiv.org/abs/2504.05506)Cited by:[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px1.p1.1)\.
- Masryet al\.\(2023\)A\. Masry, P\. Kavehzadeh, X\. L\. Do, E\. Hoque, and S\. JotyUniChart: a universal vision\-language pretrained model for chart comprehension and reasoning\.External Links:2305\.14761,[Link](https://arxiv.org/abs/2305.14761)Cited by:[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px1.p1.1)\.
- Masryet al\.\(2022\)A\. Masry, D\. X\. Long, J\. Q\. Tan, S\. Joty, and E\. HoqueChartQA: a benchmark for question answering about charts with visual and logical reasoning\.InFindings of the Association for Computational Linguistics: ACL 2022,S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),Dublin, Ireland,pp\. 2263–2279\.External Links:[Link](https://aclanthology.org/2022.findings-acl.177/),[Document](https://dx.doi.org/10.18653/v1/2022.findings-acl.177)Cited by:[§1](https://arxiv.org/html/2609.13745#S1.p1.1),[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px1.p1.1)\.
- Menget al\.\(2023\)K\. Meng, D\. Bau, A\. Andonian, and Y\. BelinkovLocating and editing factual associations in gpt\.External Links:2202\.05262,[Link](https://arxiv.org/abs/2202.05262)Cited by:[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px2.p1.1)\.
- Methaniet al\.\(2020\)N\. Methani, P\. Ganguly, M\. M\. Khapra, and P\. KumarPlotQA: reasoning over scientific plots\.External Links:1909\.00997,[Link](https://arxiv.org/abs/1909.00997)Cited by:[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px1.p1.1)\.
- Mukherjeeet al\.\(2026\)K\. Mukherjee, D\. Ren, D\. Moritz, and Y\. AssogbaEncQA: benchmarking vision\-language models on visual encodings for charts\.IEEE Transactions on Visualization and Computer Graphics32\(1\),pp\. 648–658\.External Links:ISSN 2160\-9306,[Link](http://dx.doi.org/10.1109/TVCG.2025.3634249),[Document](https://dx.doi.org/10.1109/tvcg.2025.3634249)Cited by:[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px1.p1.1)\.
- Palitet al\.\(2023\)V\. Palit, R\. Pandey, A\. Arora, and P\. P\. LiangTowards vision\-language mechanistic interpretability: a causal tracing tool for blip\.External Links:2308\.14179,[Link](https://arxiv.org/abs/2308.14179)Cited by:[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px2.p2.1)\.
- Salazaret al\.\(2026\)I\. Salazar, S\. Frank, D\. Oneata, D\. Elliott, and C\. FierroPathways of visual information flow in vision\-language models\.External Links:2607\.03358,[Link](https://arxiv.org/abs/2607.03358)Cited by:[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px2.p2.1)\.
- Tartagliniet al\.\(2025\)A\. R\. Tartaglini, S\. Grant, D\. Wurgaft, C\. Potts, and J\. E\. FanDiagnosing bottlenecks in data visualization understanding by vision\-language models\.External Links:2510\.21740,[Link](https://arxiv.org/abs/2510.21740)Cited by:[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px1.p2.1)\.
- Teamet al\.\(2026\)V\. Team, W\. Hong, X\. Gu, Z\. Pan, Z\. Yang, Y\. Wang, Y\. Wang, Y\. Yue, Y\. Wang, Y\. Wang, Y\. Wang, X\. Liu, W\. Yu, W\. Wang, W\. Li, S\. Duan, S\. Yang, R\. Lv, M\. Liu, L\. Pan, K\. Ning, J\. Ji, J\. Wang, J\. Chen, J\. Xu, J\. Zhu, J\. Cheng, J\. Qi, G\. Gan, G\. Wang, C\. Yao, Z\. Dou, Z\. Zhou, Z\. Wang, Z\. Ge, Z\. Li, Z\. Hou, Z\. Xue, Z\. Wang, Z\. Qi, Z\. He, Y\. Zhang, Y\. Liu, Y\. Cen, Y\. Li, Y\. Wang, Y\. Yang, Y\. Liu, Y\. Lu, Y\. Xu, Y\. Wang, Y\. Zhao, Y\. Wang, Y\. Xue, Y\. Xu, X\. Zhang, X\. Liu, X\. Liu, W\. Zhao, W\. Li, T\. Tong, T\. Zhang, S\. Zhang, S\. Yan, Q\. Zheng, M\. Xu, L\. Bao, lat Long long, J\. Xu, J\. Fan, J\. Qian, J\. Chen, J\. Lin, J\. Sun, H\. Zheng, H\. Wang, H\. Li, H\. Lai, H\. Xu, F\. Yang, D\. Zhang, D\. Yin, C\. Zhao, C\. Wu, B\. Shi, B\. Lv, B\. Jia, B\. Li, B\. Chen, B\. Wang, P\. Zhang, D\. Liu, B\. Xu, J\. Li, M\. Huang, Y\. Dong, and J\. TangGLM\-5v\-turbo: toward a native foundation model for multimodal agents\.External Links:2604\.26752,[Link](https://arxiv.org/abs/2604.26752)Cited by:[§1](https://arxiv.org/html/2609.13745#S1.p1.1)\.
- Wanget al\.\(2025\)W\. Wang, Z\. Gao, L\. Gu, H\. Pu, L\. Cui, X\. Wei, Z\. Liu, L\. Jing, S\. Ye, J\. Shao, Z\. Wang, Z\. Chen, H\. Zhang, G\. Yang, H\. Wang, Q\. Wei, J\. Yin, W\. Li, E\. Cui, G\. Chen, Z\. Ding, C\. Tian, Z\. Wu, J\. Xie, Z\. Li, B\. Yang, Y\. Duan, X\. Wang, Z\. Hou, H\. Hao, T\. Zhang, S\. Li, X\. Zhao, H\. Duan, N\. Deng, B\. Fu, Y\. He, Y\. Wang, C\. He, B\. Shi, J\. He, Y\. Xiong, H\. Lv, L\. Wu, W\. Shao, K\. Zhang, H\. Deng, B\. Qi, J\. Ge, Q\. Guo, W\. Zhang, S\. Zhang, M\. Cao, J\. Lin, K\. Tang, J\. Gao, H\. Huang, Y\. Gu, C\. Lyu, H\. Tang, R\. Wang, H\. Lv, W\. Ouyang, L\. Wang, M\. Dou, X\. Zhu, T\. Lu, D\. Lin, J\. Dai, W\. Su, B\. Zhou, K\. Chen, Y\. Qiao, W\. Wang, and G\. LuoInternVL3\.5: advancing open\-source multimodal models in versatility, reasoning, and efficiency\.External Links:2508\.18265,[Link](https://arxiv.org/abs/2508.18265)Cited by:[§1](https://arxiv.org/html/2609.13745#S1.p1.1),[§4\.2](https://arxiv.org/html/2609.13745#S4.SS2.p1.1)\.
- Wanget al\.\(2024\)Z\. Wang, M\. Xia, L\. He, H\. Chen, Y\. Liu, R\. Zhu, K\. Liang, X\. Wu, H\. Liu, S\. Malladi, A\. Chevalier, S\. Arora, and D\. ChenCharXiv: charting gaps in realistic chart understanding in multimodal llms\.External Links:2406\.18521,[Link](https://arxiv.org/abs/2406.18521)Cited by:[§1](https://arxiv.org/html/2609.13745#S1.p1.1),[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px1.p1.1)\.
- Xuet al\.\(2024\)Z\. Xu, S\. Du, Y\. Qi, C\. Xu, C\. Yuan, and J\. GuoChartBench: a benchmark for complex visual reasoning in charts\.External Links:2312\.15915,[Link](https://arxiv.org/abs/2312.15915)Cited by:[§1](https://arxiv.org/html/2609.13745#S1.p1.1),[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px1.p1.1)\.
- Zhang and Nanda \(2024\)F\. Zhang and N\. NandaTowards best practices of activation patching in language models: metrics and methods\.External Links:2309\.16042,[Link](https://arxiv.org/abs/2309.16042)Cited by:[§2](https://arxiv.org/html/2609.13745#S2.SS0.SSS0.Px2.p1.1)\.

## Appendix AAppendix

### A\.1Chart Value Reading Data Generation Details

X\-label pseudowords \(2,493 chart\-level occurrences\)Series/legend pseudowords \(1,150 chart\-level occurrences\)TokenCountTokenCountTokenCountTokenCountTokenCountTokenCountBebe61Beha42Beja55Cag52Bahe56Boz43Cib40Ciha46Daj43Dega54Ceh48Cul37Deje59Fece41Fehe38Fiha51Dac46Dag49Gaz60Gor49Guh48Haj57Fafa42Fup38Hav47Higa43Jide55Jof53Geh56Gibe43Jov57Kede42Kice36Kike54Haf39Hece54Lad46Lede51Lefe52Maga55Jec45Juz42Mece51Meka34Ner34Nica60Kul52Lica46Nin51Peb56Peka57Pir40Moc50Nige60Reca51Rof48Rug41Sabe65Puj54Ruc39Sebe53Suf47Tas40Tig48Seb54Tiga51Tum43Vig156Zida62Zif69Vije49Zede57

Table 7:Chart\-level occurrence counts of the pseudowords used in the 500 unique source charts\. We report counts at the source\-chart level rather than the 5,932\-query level because each chart is repeated once for every target bar, which would otherwise overweight charts containing more bars\. The x\-label vocabulary contains 48 pseudowords and the series/legend vocabulary contains 24 pseudowords\.We show the distributions of the 500 synthesized charts and 5932 chart value reading samples in Table[7](https://arxiv.org/html/2609.13745#A1.T7)and Figure[10](https://arxiv.org/html/2609.13745#A1.F10)\.

### A\.2Evaluation Results on Original and Counterfactal Chart Value Reading Full results

ModelScopeCounterfactualFFFTTFTTOri\. Acc\.CF Acc\.BothConsistencyQwen2\.5\-VL\-7BTargetAxis scale14221516427297\.5772\.0572\.02–Bar height10134134565497\.5797\.5795\.31–Legend swap13131140564897\.5797\.4295\.21–X\-label swap12132122566697\.5797\.7495\.52–Non\-targetBar height1242014577497\.5797\.6797\.3499\.43X\-label swap13777578197\.5797\.5797\.4599\.76Legend swap14133578597\.5797\.5797\.5299\.90InternVL3\.5\-8BTargetAxis scale14517731503997\.2785\.2384\.95–Bar height7092108566297\.2797\.0095\.45–Legend swap33129133563797\.2797\.2095\.03–X\-label swap43119131563997\.2797\.0795\.06–Non\-targetBar height1431919575197\.2797\.2796\.9599\.36X\-label swap1412115575597\.2797\.3797\.0299\.39Legend swap1431915575597\.2797\.3497\.0299\.43Table 8:Behavioral transitions between the original and image\-counterfactual charts overN=5,932N=5\{,\}932fixed\-prompt pairs per condition\. F and T denote strict incorrectness and correctness, respectively, on the original and counterfactual endpoints; thus, TF denotes an originally correct but counterfactually incorrect pair\. Ori\. Acc\. and CF Acc\. are the endpoint accuracies\. Both isNTT/NN\_\{\\mathrm\{TT\}\}/N\. For non\-target edits, Consistency is\(NFF\+NTT\)/N\(N\_\{\\mathrm\{FF\}\}\+N\_\{\\mathrm\{TT\}\}\)/N, measuring whether strict correctness is preserved after an edit that leaves the gold answer unchanged\. Percentages are reported in the final four columns\.Table[8](https://arxiv.org/html/2609.13745#A1.T8)reports the complete strict\-correctness transitions between the original and counterfactual charts\. We denote each pair by its correctness state on the original and counterfactual endpoints: for example, TF indicates that the model is strictly correct on the original chart but incorrect on the counterfactual chart\. For target\-relevant edits,*Both*is the proportion of pairs for which both endpoints are answered correctly,NTT/NN\_\{\\mathrm\{TT\}\}/N\. For non\-target edits, whose gold answer remains unchanged, we additionally report correctness consistency,\(NFF\+NTT\)/N\(N\_\{\\mathrm\{FF\}\}\+N\_\{\\mathrm\{TT\}\}\)/N\.

Legend edits have additional feasibility constraints\. A target\-relevant legend swap requires at least two series: 5,732 of the 5,932 queries admit an actual swap, while 200 single\-series queries are no\-op pairs\. A non\-target legend swap requires two series other than the target: 3,438 queries from three\-series charts admit an actual edit, while the remaining 2,494 pairs are pixel\-identical no\-ops\. The aggregate legend rows in Table[8](https://arxiv.org/html/2609.13745#A1.T8)include these no\-op pairs and should therefore be interpreted with this qualification\. For correctly answered pixel\-identical pairs, the absolute change in the gold\-answer log probability satisfies\|Δ​log⁡p⁡\(y⋆\)\|<10−3\\lvert\\Delta\\log p\(y^\{\\star\}\)\\rvert<10^\{\-3\}\.

\(a\) Parameter sampling

num\_series=sample\(\{1,2,3\}\)

num\_categories=sample\(\{4,5,6\}\)

tick\_step=sample\(\{5,10\}\)

num\_ticks=sample\(\{5,\.\.\.,10\}\)

y\_max=tick\_step\*num\_ticks

requirey\_max<=60

v\_min=ceil\(0\.08\*y\_max\)

v\_max=floor\(0\.92\*y\_max\)

values=sample\_integers\(

shape=\(num\_series,num\_categories\),

range=\[v\_min,v\_max\]\)

labels=sample\_unique\_pseudowords\(\)

colors=sample\_distinct\_colors\(\)

legend=Noneifnum\_series==1\\

elsesample\_fixed\_legend\_layout\(\)

outputs=\[\(448,64\),\(1344,192\)\]

\(b\) Fixed\-layout rendering

fig=plt\.figure\(figsize=\(7,7\),dpi=dpi\)

ax=fig\.add\_axes\(\[\.125,\.165,\.835,\.700\]\)

forsinrange\(num\_series\):

xpos=grouped\_bar\_positions\(

s,num\_series,num\_categories\)

ax\.bar\(xpos,values\[s\],

width=group\_width/num\_series,

color=colors\[s\],

edgecolor="none"\)

ax\.set\_xlim\(\-0\.6,num\_categories\-0\.4\)

ax\.set\_ylim\(0,y\_max\)

ax\.set\_xticks\(category\_positions\)

ax\.set\_yticks\(range\(0,y\_max\+1,tick\_step\)\)

ax\.set\_xticklabels\(\[\]\)

ax\.set\_yticklabels\(\[\]\)

draw\_ticks\_at\_fixed\_slots\(ax,labels\)

draw\_legend\_at\_fixed\_slot\(fig,legend\)

ax\.grid\(False\)

ax\.minorticks\_off\(\)

fig\.savefig\(path,dpi=dpi,

bbox\_inches=None,

pad\_inches=0\)

Figure 9:Controlled vertical\-bar chart generation template\.Chart structure, integer bar values, labels, colors, axis scale, and legend configuration are sampled explicitly\. Rendering uses a fixed canvas, axes rectangle, grouped\-bar geometry, and predefined text slots\. The paired low\- and high\-resolution images share all semantic and normalized layout parameters and differ only in DPI\. Automatic tick\-label placement, automatic legends, tight layout, and tight bounding\-box cropping are disabled\.![Refer to caption](https://arxiv.org/html/2609.13745v1/appendix_realized_sample_distribution.png)Figure 10:Realized distribution of the controlled dataset\. Panels \(a\)–\(c\) aggregate the 500 unique source charts, whereas panels \(d\)–\(f\) aggregate the 5,932 read\-value queries derived from these charts\. Panel \(c\) reports the number of source charts for each realized pair of tick step and number of tick intervals, whereymax=tick step×Ky\_\{\\max\}=\\text\{tick step\}\\times K\. The lower frequencies at category positions 5–6 and series positions 2–3 reflect structural availability: not every chart contains six categories or three series\.

## Appendix BFormal Definitions and Computation of Statistical Metrics

This section formalizes the quantities used in the main analysis and states the order in which they are computed\. Unless otherwise noted, a statistic is first computed for each query and is then aggregated across queries\. Leti∈\{1,…,N\}i\\in\\\{1,\\ldots,N\\\}index queries, letk∈\{1,…,K\}k\\in\\\{1,\\ldots,K\\\}index intervention conditions, and letg⁡\(i\)g\(i\)be the source chart associated with queryii\. A source chart can contribute several series–category queries\.

### B\.1Answer\-sequence score and answer margin

##### Definition\.

Letaioa\_\{i\}^\{o\}andaica\_\{i\}^\{c\}be the original and counterfactual gold answer sequences\. For a runx∈\{o,c,p\}x\\in\\\{o,c,p\\\}, denoting the original, counterfactual, and patched computations, respectively, the teacher\-forced score of an answer sequenceaais

Six​\(a\)=∑t=1T⁡\(a\)log⁡Pθ​\(at∣xi,qi,a<t\)\.S\_\{i\}^\{x\}\(a\)=\\sum\_\{t=1\}^\{T\(a\)\}\\log P\_\{\\theta\}\\bigl\(a\_\{t\}\\mid x\_\{i\},q\_\{i\},a\_\{<t\}\\bigr\)\.\(14\)The answer\-preference margin in runxxis

Mix=Six​\(aio\)−Six​\(aic\)\.M\_\{i\}^\{x\}=S\_\{i\}^\{x\}\(a\_\{i\}^\{o\}\)\-S\_\{i\}^\{x\}\(a\_\{i\}^\{c\}\)\.\(15\)

##### Explanation\.

An integer answer can contain more than one output token\. Scoring the complete answer sequence avoids reducing a multi\-token number to its first token\. The margin compares the only two answers relevant to the paired intervention: the original gold answer and the counterfactual gold answer\. A positive margin favors the original answer, while a negative margin favors the counterfactual answer\.

### B\.2Normalized restoration

##### Definition\.

For queryii, normalized restoration is

Ri=Mip−MicMio−Mic\.R\_\{i\}=\\frac\{M\_\{i\}^\{p\}\-M\_\{i\}^\{c\}\}\{M\_\{i\}^\{o\}\-M\_\{i\}^\{c\}\}\.\(16\)
Restoration values are not clipped to\[0,1\]\[0,1\]\.

##### Explanation\.

The denominator is the full answer\-preference change between the counterfactual and original runs\. The numerator is the part of that change recovered by patching\. Thus,Ri=0R\_\{i\}=0means that patching leaves the counterfactual preference unchanged, whileRi=1R\_\{i\}=1means that the full original–counterfactual margin gap is restored\. A value above one indicates overshooting, and a negative value indicates movement further toward the counterfactual answer\. Normalization makes effects more comparable across queries and models with different raw log\-probability scales\.

### B\.3Projected\-site, layer\-wise, and prompt restoration

##### Definition\.

For image interventionkk, letΩi,k\\Omega\_\{i,k\}be its pixel\-level region of interest \(ROI\), and let

𝒥i,km=𝒯m​\(Ωi,k\)\\mathcal\{J\}\_\{i,k\}^\{m\}=\\mathcal\{T\}\_\{m\}\(\\Omega\_\{i,k\}\)\(17\)be the visual\-token set selected by spatial mapping rulemm\. Projected\-site restoration is the normalized restoration obtained by replacing the selected counterfactual projected tokens with their original counterparts:

Ri,kproj,m=Ri​\(CLOSE\\displaystyle R\_\{i,k\}^\{\\mathrm\{proj\},m\}=R\_\{i\}\\bigl\(hi,projc​\[𝒥i,km\]\\displaystyle h\_\{i,\\mathrm\{proj\}\}^\{c\}\[\\mathcal\{J\}\_\{i,k\}^\{m\}\]\(18\)OPEN←hi,projo​\[𝒥i,km\]\)\.\\displaystyle\\leftarrow h\_\{i,\\mathrm\{proj\}\}^\{o\}\[\\mathcal\{J\}\_\{i,k\}^\{m\}\]\\bigr\)\.At language layerℓ\\ell, image\-token restoration is denoted

Ri,k,ℓm=Ri​\(CLOSE\\displaystyle R\_\{i,k,\\ell\}^\{m\}=R\_\{i\}\\bigl\(hi,ℓc​\[𝒥i,km\]\\displaystyle h\_\{i,\\ell\}^\{c\}\[\\mathcal\{J\}\_\{i,k\}^\{m\}\]\(19\)OPEN←hi,ℓo​\[𝒥i,km\]\)\.\\displaystyle\\leftarrow h\_\{i,\\ell\}^\{o\}\[\\mathcal\{J\}\_\{i,k\}^\{m\}\]\\bigr\)\.For prompt interventionkk, let𝒫i,k\\mathcal\{P\}\_\{i,k\}be the changed prompt\-token positions\. Prompt restoration at layerℓ\\ellis

Ri,k,ℓprompt=Ri​\(CLOSE\\displaystyle R\_\{i,k,\\ell\}^\{\\mathrm\{prompt\}\}=R\_\{i\}\\bigl\(hi,ℓc​\[𝒫i,k\]\\displaystyle h\_\{i,\\ell\}^\{c\}\[\\mathcal\{P\}\_\{i,k\}\]\(20\)OPEN←hi,ℓo​\[𝒫i,k\]\)\.\\displaystyle\\leftarrow h\_\{i,\\ell\}^\{o\}\[\\mathcal\{P\}\_\{i,k\}\]\\bigr\)\.

##### Explanation\.

Projected\-site restoration asks how much of the original answer preference can be recovered using only the visual tokens associated with one semantic region, such as the changed bar top, invariant bar body, legend entries, x\-axis labels, or y\-axis scale\. Layer\-wise restoration repeats the same intervention at different language layers to measure where the local state remains causally recoverable\. Prompt interventions patch category or series token positions while keeping the image fixed; they therefore do not have a projected\-image condition\.

### B\.4Query\-weighted mean

##### Definition\.

For a query\-level statisticZiZ\_\{i\}, the query\-weighted estimate is

μ^query=1N​∑i=1NZi\.\\widehat\{\\mu\}\_\{\\mathrm\{query\}\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}Z\_\{i\}\.\(21\)

##### Explanation\.

This estimate gives every series–category question equal weight\. It answers: if one query is drawn from the evaluation pool, what effect is expected on average? It is the primary estimand because each patching record corresponds to one concrete value\-reading query\.

### B\.5Query\-level bootstrap confidence interval

##### Definition\.

For bootstrap replicateb∈\{1,…,B\}b\\in\\\{1,\\ldots,B\\\}, sampleNNquery indices with replacement and compute

μ^∗\(b\)=1N​∑j=1NZib,j∗\.\\widehat\{\\mu\}^\{\*\(b\)\}=\\frac\{1\}\{N\}\\sum\_\{j=1\}^\{N\}Z\_\{i\_\{b,j\}^\{\*\}\}\.\(22\)We useB=5,000B=5\{,\}000replicates with random seed 42\. Define the bootstrap sample compactly as

ℬμ=\{μ^∗\(b\):b=1,…,B\}\.\\mathcal\{B\}\_\{\\mu\}=\\\{\\widehat\{\\mu\}^\{\*\(b\)\}:b=1,\\ldots,B\\\}\.\(23\)The percentile interval is

CI95%=\[Q0\.025​\(ℬμ\),Q0\.975​\(ℬμ\)\],\\displaystyle\\mathrm\{CI\}\_\{95\\%\}=\\bigl\[Q\_\{0\.025\}\(\\mathcal\{B\}\_\{\\mu\}\),Q\_\{0\.975\}\(\\mathcal\{B\}\_\{\\mu\}\)\\bigr\],\(24\)whereQαQ\_\{\\alpha\}is the empiricalα\\alpha\-quantile\.

##### Explanation\.

The bootstrap estimates how much the reported mean would vary if a similar set of queries were sampled again\. It does not require the restoration distribution to be Gaussian, which is useful because normalized restoration can be negative or greater than one\. For an AUC comparison, the complete AUC is first computed for every query; the resulting query\-level AUC values, rather than individual layer points, are then resampled\.

### B\.6Paired condition and ROI contrasts

##### Definition\.

Suppose conditionsAAandBBuse the same query keys\. Their within\-query difference is

ΔiA−B=ZiA−ZiB\.\\Delta\_\{i\}^\{A\-B\}=Z\_\{i\}^\{A\}\-Z\_\{i\}^\{B\}\.\(25\)The paired mean difference is

Δ^A−B=1N​∑i=1NΔiA−B\.\\widehat\{\\Delta\}^\{A\-B\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\Delta\_\{i\}^\{A\-B\}\.\(26\)The positive\-direction fraction is

π^\+A−B=1N∑i=1N𝟏\[ΔiA−B\>0\]\.\\widehat\{\\pi\}\_\{\+\}^\{A\-B\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbf\{1\}\[\\Delta\_\{i\}^\{A\-B\}\>0\]\.\(27\)Confidence intervals are formed by bootstrapping the paired differences\{ΔiA−B\}i=1N\\\{\\Delta\_\{i\}^\{A\-B\}\\\}\_\{i=1\}^\{N\}rather than the two conditions independently\.

##### Explanation\.

Pairing removes variation caused by different charts, targets, or counterfactual edits\. Bar difference and bar intersection, for example, use the same original–counterfactual image pair and differ only in the patched ROI\. Their paired contrast therefore asks whether the changed bar top restores more information than the invariant body on the same query\. The paired mean measures the average size of this advantage, while the positive\-direction fraction measures how consistently the same direction holds across individual queries\.

### B\.7Source\-chart cluster bootstrap

##### Definition\.

Let

ℐc=\{i:g⁡\(i\)=c\},nc=\|ℐc\|\\mathcal\{I\}\_\{c\}=\\\{i:g\(i\)=c\\\},\\qquad n\_\{c\}=\\lvert\\mathcal\{I\}\_\{c\}\\rvert\(28\)be the queries derived from source chartcc\. In replicatebb, sampleCCsource\-chart identifiers with replacement and include all queries belonging to each sampled chart\. The query\-weighted replicate is

μ^cl∗\(b\)=∑j=1C∑i∈ℐcb,j∗Zi∑j=1Cncb,j∗\.\\displaystyle\\widehat\{\\mu\}\_\{\\mathrm\{cl\}\}^\{\*\(b\)\}=\\frac\{\\sum\_\{j=1\}^\{C\}\\sum\_\{i\\in\\mathcal\{I\}\_\{c\_\{b,j\}^\{\*\}\}\}Z\_\{i\}\}\{\\sum\_\{j=1\}^\{C\}n\_\{c\_\{b,j\}^\{\*\}\}\}\.\(29\)The cluster\-bootstrap interval is obtained from the 2\.5th and 97\.5th percentiles ofB=5,000B=5\{,\}000such replicates\.

##### Explanation\.

Queries from the same source chart share layout, colors, labels, axis configuration, and values\. Treating them as fully independent could make an uncertainty interval too narrow\. Cluster bootstrap resamples whole charts and keeps all associated queries together\. It preserves the query\-weighted target of estimation while accounting for dependence among queries from the same chart\. For a paired comparison,ZiZ\_\{i\}is replaced withΔiA−B\\Delta\_\{i\}^\{A\-B\}\.

### B\.8Chart\-equal estimate

##### Definition\.

First compute the mean within source chartcc:

Z¯c=1nc​∑i∈ℐcZi\.\\overline\{Z\}\_\{c\}=\\frac\{1\}\{n\_\{c\}\}\\sum\_\{i\\in\\mathcal\{I\}\_\{c\}\}Z\_\{i\}\.\(30\)The chart\-equal estimate is

μ^chart=1C​∑c=1CZ¯c\.\\widehat\{\\mu\}\_\{\\mathrm\{chart\}\}=\\frac\{1\}\{C\}\\sum\_\{c=1\}^\{C\}\\overline\{Z\}\_\{c\}\.\(31\)

##### Explanation\.

The query\-weighted mean gives more influence to charts that contribute more target bars\. The chart\-equal estimate instead gives every source chart the same total weight\. Agreement between the two estimates shows that a result is not driven only by a small number of charts that happen to generate many queries\.

### B\.9ROI area and selected\-token controls

##### Definition\.

For ROIrr, normalized pixel area is

Ai,r=1H​W∑u=1H∑v=1W𝟏\[\(u,v\)∈Ωi,r\]\.A\_\{i,r\}=\\frac\{1\}\{HW\}\\sum\_\{u=1\}^\{H\}\\sum\_\{v=1\}^\{W\}\\mathbf\{1\}\[\(u,v\)\\in\\Omega\_\{i,r\}\]\.\(32\)Under mapping rulemm, the number of selected visual tokens is

Ni,rm=\|𝒯m​\(Ωi,r\)\|\.N\_\{i,r\}^\{m\}=\\lvert\\mathcal\{T\}\_\{m\}\(\\Omega\_\{i,r\}\)\\rvert\.\(33\)For the main changed\-top comparison, define

𝒮tokm=\{i:Ni,diffm≤Ni,interm\}\.\\mathcal\{S\}\_\{\\mathrm\{tok\}\}^\{m\}=\\\{i:N\_\{i,\\mathrm\{diff\}\}^\{m\}\\leq N\_\{i,\\mathrm\{inter\}\}^\{m\}\\\}\.\(34\)The restricted paired effect is

Δ^tokm=1\|𝒮tokm\|​∑i∈𝒮tokm\(Ri,diffm−Ri,interm\)\.\\displaystyle\\widehat\{\\Delta\}\_\{\\mathrm\{tok\}\}^\{m\}=\\frac\{1\}\{\\lvert\\mathcal\{S\}\_\{\\mathrm\{tok\}\}^\{m\}\\rvert\}\\sum\_\{i\\in\\mathcal\{S\}\_\{\\mathrm\{tok\}\}^\{m\}\}\\bigl\(R\_\{i,\\mathrm\{diff\}\}^\{m\}\-R\_\{i,\\mathrm\{inter\}\}^\{m\}\\bigr\)\.\(35\)

##### Explanation\.

Patching more tokens can mechanically increase restoration because more of the original state is inserted\. The restricted comparison keeps only queries for which the changed bar\-top ROI selects no more tokens than the invariant body\. If the changed\-top advantage remains, it cannot be explained simply by patching a larger number of tokens\. Pixel area and token count are both useful because a small pixel region can intersect several token cells\.

### B\.10Fraction of the whole\-bar effect

##### Definition\.

For queries with\|Ri,union\|≥ϵ\\lvert R\_\{i,\\mathrm\{union\}\}\\rvert\\geq\\epsilon, define

Qi,diff\\displaystyle Q\_\{i,\\mathrm\{diff\}\}=Ri,diffRi,union,\\displaystyle=\\frac\{R\_\{i,\\mathrm\{diff\}\}\}\{R\_\{i,\\mathrm\{union\}\}\},\(36\)Qi,inter\\displaystyle Q\_\{i,\\mathrm\{inter\}\}=Ri,interRi,union\.\\displaystyle=\\frac\{R\_\{i,\\mathrm\{inter\}\}\}\{R\_\{i,\\mathrm\{union\}\}\}\.\(37\)The analysis reports the sample mean and median of these per\-query ratios\.

##### Explanation\.

The union ROI covers the full affected bar\. These ratios ask how much of that whole\-bar effect is already captured by the changed top or by the invariant body\. A changed\-top\-to\-union ratio near one means that the narrow changed band restores almost as much answer preference as the whole\-bar patch\. Reporting both mean and median makes the summary less dependent on a few unusually large ratios\.

### B\.11Normalized layer depth

##### Definition\.

For a model withLLlanguage layers indexed byℓ∈\{0,…,L−1\}\\ell\\in\\\{0,\\ldots,L\-1\\\}, normalized depth is

dℓ=ℓL−1\.d\_\{\\ell\}=\\frac\{\\ell\}\{L\-1\}\.\(38\)

##### Explanation\.

The evaluated models have different layer counts, so raw layer indices are not directly comparable\. Normalized depth maps the first layer to zero and the final layer to one\. It allows transition locations to be compared as fractions of total network depth, but it does not imply that two layers at the same normalized depth perform identical computations\.

### B\.12Raw layer\-profile AUC

##### Definition\.

LetRi,k,ℓR\_\{i,k,\\ell\}be single\-layer restoration\. The per\-query raw AUC is computed using the trapezoidal rule:

AUCi,kraw=∑ℓ=0L−2\\displaystyle\\mathrm\{AUC\}\_\{i,k\}^\{\\mathrm\{raw\}\}=\\sum\_\{\\ell=0\}^\{L\-2\}Ri,k,ℓ\+Ri,k,ℓ\+12\\displaystyle\\frac\{R\_\{i,k,\\ell\}\+R\_\{i,k,\\ell\+1\}\}\{2\}\(39\)×\(dℓ\+1−dℓ\)\.\\displaystyle\\times\(d\_\{\\ell\+1\}\-d\_\{\\ell\}\)\.A paired AUC advantage of conditionAAover conditionBBis

Δi,AUCA−B=AUCi,Araw−AUCi,Braw\.\\Delta\_\{i,\\mathrm\{AUC\}\}^\{A\-B\}=\\mathrm\{AUC\}\_\{i,A\}^\{\\mathrm\{raw\}\}\-\\mathrm\{AUC\}\_\{i,B\}^\{\\mathrm\{raw\}\}\.\(40\)

##### Explanation\.

Raw AUC summarizes the total restoration accumulated over the full network depth\. It is large when an ROI starts with a strong restoration effect, when that effect remains accessible across many layers, or both\. It therefore measures cumulative recoverable information, but it does not separate initial magnitude from persistence\. AUC is computed for each query before averaging, pairing, or bootstrapping; this is the quantity referred to as the raw layer\-profile AUC in the prompt\-label versus prompt\-series comparison\.

### B\.13Retention profile and retention AUC

##### Definition\.

For queries whose layer\-0 restoration is nonzero at numerical precision, define

ℰk=\{i:\|Ri,k,0\|≥ϵ\}\.\\mathcal\{E\}\_\{k\}=\\\{i:\\lvert R\_\{i,k,0\}\\rvert\\geq\\epsilon\\\}\.\(41\)For each eligible query, the retention profile is

ρi,k,ℓ=Ri,k,ℓRi,k,0\.\\rho\_\{i,k,\\ell\}=\\frac\{R\_\{i,k,\\ell\}\}\{R\_\{i,k,0\}\}\.\(42\)Its per\-query retention AUC is

AUCi,kret=∑ℓ=0L−2\\displaystyle\\mathrm\{AUC\}\_\{i,k\}^\{\\mathrm\{ret\}\}=\\sum\_\{\\ell=0\}^\{L\-2\}ρi,k,ℓ\+ρi,k,ℓ\+12\\displaystyle\\frac\{\\rho\_\{i,k,\\ell\}\+\\rho\_\{i,k,\\ell\+1\}\}\{2\}\(43\)×\(dℓ\+1−dℓ\)\.\\displaystyle\\times\(d\_\{\\ell\+1\}\-d\_\{\\ell\}\)\.

##### Explanation\.

Two ROIs can begin with very different restoration magnitudes\. Dividing each query’s curve by its own layer\-0 value sets every eligible curve to one at layer zero, so the remaining quantity measures the fraction of the initial local effect that remains recoverable\. Raw AUC asks how much recoverable information accumulates across depth; retention AUC asks how slowly the initially available effect decays\. These two metrics must be reported together because a weak effect can persist for a long time and a strong early effect can disappear quickly\.

### B\.14Relative half\-life

##### Definition\.

The mean retention curve for conditionkkis

ρ¯k,ℓ=1\|ℰk\|​∑i∈ℰkρi,k,ℓ\.\\overline\{\\rho\}\_\{k,\\ell\}=\\frac\{1\}\{\\lvert\\mathcal\{E\}\_\{k\}\\rvert\}\\sum\_\{i\\in\\mathcal\{E\}\_\{k\}\}\\rho\_\{i,k,\\ell\}\.\(44\)The half\-life layer is the first native layer position at which this mean curve reaches or falls below one half:

ℓ1/2k=min⁡\{ℓ:ρ¯k,ℓ≤0\.5\}\.\\ell\_\{1/2\}^\{k\}=\\min\\\{\\ell:\\overline\{\\rho\}\_\{k,\\ell\}\\leq 0\.5\\\}\.\(45\)Relative half\-life is

Hk=dℓ1/2k=ℓ1/2kL−1\.H\_\{k\}=d\_\{\\ell\_\{1/2\}^\{k\}\}=\\frac\{\\ell\_\{1/2\}^\{k\}\}\{L\-1\}\.\(46\)If the mean curve never reaches0\.50\.5, half\-life is reported as not reached\.

##### Explanation\.

Relative half\-life provides one readable location for the decay of a retention curve: it is the relative network depth at which less than half of the initial local effect remains\. The reported values align with native normalized layer positions; for example,22/27≈0\.81522/27\\approx 0\.815for a 28\-layer model and23/35≈0\.65723/35\\approx 0\.657for a 36\-layer model\. Half\-life is only a summary of the curve\. It is not the exact layer at which the model performs a computation, because information can be transformed or redistributed and thereby become inaccessible to a particular local patch\.

### B\.15Contiguous layer\-window patching

##### Definition\.

For widthw∈\{2,4\}w\\in\\\{2,4\\\}and starting layers∈\{0,…,L−w\}s\\in\\\{0,\\ldots,L\-w\\\}, define

𝒲s,w=\{s,s\+1,…,s\+w−1\}\.\\mathcal\{W\}\_\{s,w\}=\\\{s,s\+1,\\ldots,s\+w\-1\\\}\.\(47\)All selected token states in this window are replaced jointly in one forward pass\. We denote the resulting normalized restoration by

Ri,k,s\(w\)=Ri​\(Patch⁡\(𝒥i,k,𝒲s,w\)\)\.R\_\{i,k,s\}^\{\(w\)\}=R\_\{i\}\\bigl\(\\operatorname\{Patch\}\(\\mathcal\{J\}\_\{i,k\},\\mathcal\{W\}\_\{s,w\}\)\\bigr\)\.\(48\)Windows use stride one\. Their normalized center depth is

ds,wcenter=s\+\(w−1\)/2L−1\.d\_\{s,w\}^\{\\mathrm\{center\}\}=\\frac\{s\+\(w\-1\)/2\}\{L\-1\}\.\(49\)

##### Explanation\.

A representation can be distributed across neighboring layers, so a single\-layer intervention may underestimate how long it remains recoverable\. Window patching jointly restores a short contiguous range to test whether the ordering and transition regions remain visible\. The result is produced by a new multi\-layer forward intervention; it is not the sum or average of the corresponding single\-layer effects\. A larger window effect is therefore interpreted as accumulated recovery, not as evidence that every layer in the window performs the same operation\.

### B\.16Spatial\-expansion Gain and Within\-Model Profile Agreement

This analysis compares the two spatial ROI\-to\-token mappings evaluated for Qwen2\.5\-VL: strict any\-overlap mapping and merge\-closed3×33\{\\times\}3coarse mapping\. The comparison contains two distinct quantities\. Projected\-site gain measures a change in restoration magnitude at the projected image\-token site, whereas within\-model profile correlation measures whether the two mappings produce similar restoration trajectories across the model’s native language layers\.

##### Definition: projected\-site restoration gain\.

Let

m∈\{any,coarse\}m\\in\\\{\\mathrm\{any\},\\mathrm\{coarse\}\\\}denote the spatial mapping rule\. For an image interventionkk, queryii, and mappingmm, projected\-site restorationRi,kproj,mR\_\{i,k\}^\{\\mathrm\{proj\},m\}is defined in Equation[18](https://arxiv.org/html/2609.13745#A2.E18)\. The per\-query gain from coarse mapping is

Gi,kproj=Ri,kproj,coarse−Ri,kproj,any\.G\_\{i,k\}^\{\\mathrm\{proj\}\}=R\_\{i,k\}^\{\\mathrm\{proj\},\\mathrm\{coarse\}\}\-R\_\{i,k\}^\{\\mathrm\{proj\},\\mathrm\{any\}\}\.\(50\)Its paired mean is

G^kproj=1N​∑i=1NGi,kproj\.\\widehat\{G\}\_\{k\}^\{\\mathrm\{proj\}\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}G\_\{i,k\}^\{\\mathrm\{proj\}\}\.\(51\)

##### Explanation\.

This quantity compares the two mappings at one fixed patching site: the projected image\-token representation before subsequent language\-layer processing\. A positive value means that replacing the coarse ROI\-selected projected tokens restores more of the original answer preference than replacing the tokens selected by strict any\-overlap mapping\. It measures a difference in restoration magnitude, not a difference in the shape or timing of a layer\-wise profile\. Prompt interventions have no projected\-site gain because their patched supports are prompt\-token positions rather than image ROIs\.

##### Definition: mapping\-specific mean layer profiles\.

For an image interventionkk, the layer\-wise restoration under mappingmmisRi,k,ℓmR\_\{i,k,\\ell\}^\{m\}, as defined in Equation[19](https://arxiv.org/html/2609.13745#A2.E19)\. At native language layerℓ\\ell, its query\-averaged profile value is

R¯k,ℓm=1N​∑i=1NRi,k,ℓm\.\\overline\{R\}\_\{k,\\ell\}^\{m\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}R\_\{i,k,\\ell\}^\{m\}\.\(52\)The complete mean profile is

𝐑¯km=\(R¯k,0m,…,R¯k,L−1m\)\.\\overline\{\\mathbf\{R\}\}\_\{k\}^\{m\}=\\left\(\\overline\{R\}\_\{k,0\}^\{m\},\\ldots,\\overline\{R\}\_\{k,L\-1\}^\{m\}\\right\)\.\(53\)

##### Explanation\.

Each element of𝐑¯km\\overline\{\\mathbf\{R\}\}\_\{k\}^\{m\}is obtained by patching the ROI\-selected image\-token positions at one particular language layer and then averaging restoration over the synchronized queries\. The vector therefore describes how the locally patched information remains recoverable across language\-model depth\. It is separate from projected\-site restoration, which produces one value per query at the fixed projected image\-token site\.

##### Definition: within\-model profile correlation\.

Because both mappings are evaluated in the same Qwen2\.5\-VL model, they share the sameLLnative language layers\. Their profile\-shape agreement for image interventionkkis

rkmap=corrP⁡\(𝐑¯kany,𝐑¯kcoarse\),r\_\{k\}^\{\\mathrm\{map\}\}=\\operatorname\{corr\}\_\{\\mathrm\{P\}\}\\left\(\\overline\{\\mathbf\{R\}\}\_\{k\}^\{\\mathrm\{any\}\},\\overline\{\\mathbf\{R\}\}\_\{k\}^\{\\mathrm\{coarse\}\}\\right\),\(54\)wherecorrP\\operatorname\{corr\}\_\{\\mathrm\{P\}\}denotes Pearson correlation across theLLnative layer positions\. No normalized\-depth interpolation is required for this within\-model comparison\.

##### Explanation\.

The correlation measures whether the two average layer\-wise curves rise, decline, and approach zero at similar native layers\. Pearson correlation mainly captures profile shape rather than absolute magnitude\. The coarse profile can therefore have larger restoration values while still showing a correlation close to one with the any\-overlap profile\.

A high value ofrkmapr\_\{k\}^\{\\mathrm\{map\}\}does not mean that the two mappings patch the same tokens or produce equal restoration at every layer\. It means only that changing the ROI\-to\-token mapping leaves the main depth\-dependent trajectory largely unchanged\.

### B\.17Cross\-model profile and query\-level correlations

##### Definition: native mean profiles\.

For modelmmwithLmL\_\{m\}layers, the condition\-level mean restoration at native layerℓ\\ellis

R¯k,ℓ\(m\)=1N​∑i=1NRi,k,ℓ\(m\),\\overline\{R\}\_\{k,\\ell\}^\{\(m\)\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}R\_\{i,k,\\ell\}^\{\(m\)\},\(55\)and its normalized depth is

dℓ\(m\)=ℓLm−1\.d\_\{\\ell\}^\{\(m\)\}=\\frac\{\\ell\}\{L\_\{m\}\-1\}\.\(56\)

##### Definition: common grid and interpolation\.

We use the common grid

𝒟G=\{dj=j/G:j=0,…,G\},G=100\.\\mathcal\{D\}\_\{G\}=\\\{d\_\{j\}=j/G:j=0,\\ldots,G\\\},\\qquad G=100\.\(57\)Thus, both mean profiles are represented at 101 equally spaced relative depths\. For a grid pointdjd\_\{j\}lying between two adjacent native depths,dℓ\(m\)≤dj≤dℓ\+1\(m\)d\_\{\\ell\}^\{\(m\)\}\\leq d\_\{j\}\\leq d\_\{\\ell\+1\}^\{\(m\)\}, define

λj,ℓ\(m\)=dj−dℓ\(m\)dℓ\+1\(m\)−dℓ\(m\)\.\\lambda\_\{j,\\ell\}^\{\(m\)\}=\\frac\{d\_\{j\}\-d\_\{\\ell\}^\{\(m\)\}\}\{d\_\{\\ell\+1\}^\{\(m\)\}\-d\_\{\\ell\}^\{\(m\)\}\}\.\(58\)The piecewise\-linear interpolated value is

R~k\(m\)​\(dj\)=\\displaystyle\\widetilde\{R\}\_\{k\}^\{\(m\)\}\(d\_\{j\}\)=\{\}\(1−λj,ℓ\(m\)\)​R¯k,ℓ\(m\)\\displaystyle\(1\-\\lambda\_\{j,\\ell\}^\{\(m\)\}\)\\overline\{R\}\_\{k,\\ell\}^\{\(m\)\}\(59\)\+λj,ℓ\(m\)​R¯k,ℓ\+1\(m\)\.\\displaystyle\+\\lambda\_\{j,\\ell\}^\{\(m\)\}\\overline\{R\}\_\{k,\\ell\+1\}^\{\(m\)\}\.Atd0=0d\_\{0\}=0anddG=1d\_\{G\}=1, the original first\- and last\-layer values are used directly\.

##### Definition: profile correlation\.

Let

𝐑~k\(m\)=\{R~k\(m\)​\(dj\)\}j=0G\.\\widetilde\{\\mathbf\{R\}\}\_\{k\}^\{\(m\)\}=\\\{\\widetilde\{R\}\_\{k\}^\{\(m\)\}\(d\_\{j\}\)\\\}\_\{j=0\}^\{G\}\.\(60\)For modelsAAandBB, profile similarity is

rkprofile=corrP⁡\(𝐑~k\(A\),𝐑~k\(B\)\)\.r\_\{k\}^\{\\mathrm\{profile\}\}=\\operatorname\{corr\}\_\{\\mathrm\{P\}\}\\bigl\(\\widetilde\{\\mathbf\{R\}\}\_\{k\}^\{\(A\)\},\\widetilde\{\\mathbf\{R\}\}\_\{k\}^\{\(B\)\}\\bigr\)\.\(61\)The mean correlation across theKKintervention types is

r¯profile=1K​∑k=1Krkprofile\.\\overline\{r\}^\{\\mathrm\{profile\}\}=\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}r\_\{k\}^\{\\mathrm\{profile\}\}\.\(62\)

##### Explanation: what profile correlation measures\.

Each point enteringrkprofiler\_\{k\}^\{\\mathrm\{profile\}\}is a relative\-depth position on an average curve, not an individual chart\. A high value means that the two mean curves tend to stay high, decline, and approach zero at similar relative depths\. Pearson correlation is mainly a measure of curve shape: two curves can have different absolute magnitudes and still be highly correlated if they rise and fall in similar places\. The interpolation only aligns the horizontal depth axis\. It does not create an actual intermediate layer or imply a one\-to\-one functional correspondence between the two architectures\.

##### Definition: per\-query projected\-restoration correlation\.

For synchronized queries, the cross\-model sample correlation at the projected image\-token site is

rkquery=corrP⁡\(CLOSE\\displaystyle r\_\{k\}^\{\\mathrm\{query\}\}=\\operatorname\{corr\}\_\{\\mathrm\{P\}\}\\bigl\(\{Ri,kproj,A\}i=1N,\\displaystyle\\\{R\_\{i,k\}^\{\\mathrm\{proj\},A\}\\\}\_\{i=1\}^\{N\},\(63\)OPEN\{Ri,kproj,B\}i=1N\)\.\\displaystyle\\\{R\_\{i,k\}^\{\\mathrm\{proj\},B\}\\\}\_\{i=1\}^\{N\}\\bigr\)\.No layer interpolation is used here: every query contributes one projected\-site restoration value from each model\.

##### Explanation: why the two correlations can differ\.

The profile correlation first averages over all queries at each layer, so chart\-specific differences are largely smoothed out\. It asks whether the two models share the same broad population\-level depth pattern\. The query\-level correlation keeps the individual examples separate\. It asks whether a chart with unusually high restoration in one model also has unusually high restoration in the other\. The two models can therefore have a high profile correlation but a low query\-level correlation: they can agree on the average ordering and timing of information loss while disagreeing about which exact charts are most locally recoverable\. Low query\-level agreement can reflect architectural differences, visual token grids, spatial mappings, and model\-specific answer\-margin scales\. Neither correlation by itself establishes that the two models implement an identical internal mechanism\.

## Appendix CEvaluation Data Audit

This appendix reports how the activation\-patching evaluation set is formed and summarizes its realized coverage\. The sampler uses seed 42 and returns 500 queries for each of the nine analysis conditions, giving 4,500 records per model\. The visual conditions form 500 complete bundles and the prompt conditions form another 500 complete bundles\. Counting each pool\-level target once gives 1,500 selections and 1,361 distinct target\-bar IDs from 464 of the 500 source charts\. Within every condition, the two models use identical sample IDs, order, and intervention metadata\. All selected pairs have different original and counterfactual gold values, and both models answer both endpoints exactly\.

### C\.1Selection rule and evaluation scope

A pair is*cross\-model eligible*when both models answer both the original and counterfactual endpoints exactly and the two gold values differ\. This synchronized\-success rule ensures that activation restoration is measured on examples for which both endpoints are solved by both models\. The resulting analysis characterizes successful value\-reading computations; the composition checks below verify that this subset retains broad coverage over the measured chart and target factors\.

Table[9](https://arxiv.org/html/2609.13745#A3.T9)reports the image\-counterfactual selection funnel\. Axis scaling is the most selective condition, while the three local image edits retain approximately 89–91% of feasible pairs\. The primary bar\-region comparison is especially well controlled: bar union, difference, and intersection use the same 500 original–counterfactual pairs and differ only in the patched ROI\. Thus, their paired contrast does not mix different success\-selected targets\.

CounterfactualInitial pairsFeasible pairsQwen2\.5\-VL both correctInternVL3\.5 both correctCross\-model eligibleFinal sampledAxis scale5,9325,9324,2725,0393,720 \(62\.7%\)500Bar height5,9325,9325,6545,6625,408 \(91\.2%\)500Legend swap5,9325,7325,6485,6375,121 \(89\.3%\)500X\-label swap5,9325,9325,6665,6395,397 \(91\.0%\)500

Table 9:Selection funnel for image counterfactuals\. “Both correct” means that one model answers both endpoints exactly\. Cross\-model eligibility requires all four endpoint predictions to be correct and the two gold values to differ\. A target\-relevant legend swap requires at least two series, so 200 single\-series queries are excluded from its feasible denominator\. The model\-level both\-correct columns are measured on all 5,932 generated pairs\.
### C\.2Evaluation pools and shared units

A bundle contains conditions evaluated on the same original query sources, which enables within\-query comparisons\. Table[10](https://arxiv.org/html/2609.13745#A3.T10)states what is shared in each pool\.

PoolConditionsShared source relationVisual bundleBar union, bar difference, bar intersection, legend swap, x\-label swapAll five conditions use the same 500 original query IDs and target bars from 417 source charts\. The three bar conditions additionally use the exact same original–counterfactual bar\-height pairs and differ only in the ROI\. Legend and x\-label interventions apply their own edits to the same original query sources\.Prompt bundleLabel, series, label\+seriesAll three conditions use the same 500 original images, query IDs, and target bars from 418 source charts\. Each condition applies its own prompt edit while keeping the image fixed\.Axis poolAxis scaleAxis scale uses a separately sampled set of 500 query IDs from 307 source charts\.Table 10:Definition of the three evaluation pools\. The two models use identical sample IDs, order, and intervention metadata within every condition\.Table[11](https://arxiv.org/html/2609.13745#A3.T11)lists the nine analysis conditions\. The three bar rows share one bar\-height counterfactual and differ only in the ROI\. Prompt conditions patch text positions and therefore have no image ROI\.

ModalityConditionQueriesChartsTarget barsInternVL3\.5 ROI meanQwen2\.5\-VL ROI meanPoolImageAxis scale5003075003\.048%2\.309%AxisBar union5004175002\.462%2\.106%VisualBar difference5004175000\.667%0\.581%VisualBar intersection5004175001\.795%1\.524%VisualLegend swap5004175000\.918%0\.684%VisualX\-label swap5004175001\.336%1\.122%VisualPromptLabel500418500––PromptSeries500418500––PromptLabel \+ series500418500––Prompt

Table 11:Composition of the nine conditions\. ROI values are mean fractions of the rendered image area; raw pixel areas are not compared across resolutions\.The visual bundle contains 334 charts used once and 83 used twice\. The prompt bundle contains 336 charts used once and 82 used twice\. The axis pool contains 176 charts used once, 91 used twice, 26 used three times, 11 used four times, one used six times, and two used seven times\. Among the 1,500 pool\-level target selections, 1,361 target\-bar IDs are distinct, so some targets recur across pools\. At the source\-chart level, the union contains 464 charts, and the pairwise overlaps are reported in Table[13](https://arxiv.org/html/2609.13745#A3.T13)\. The main statistical analysis therefore includes source\-chart cluster bootstrap intervals and chart\-equal estimates in addition to query\-level estimates\.

PoolQueriesChartsMean/chartMax/chartVisual bundle5004171\.202Prompt bundle5004181\.202Axis scale5003071\.637Table 12:Query and source\-chart counts in the three evaluation pools\.AxisVisualPromptAxis307270275Visual270417389Prompt275389418Table 13:Source\-chart overlap among the three pools\. Diagonal entries give the number of charts in each pool\.
### C\.3Target\-value and position coverage

Figure[11](https://arxiv.org/html/2609.13745#A3.F11)summarizes the main distribution checks\. Across the three selected pools, original target values span 2–55, and normalized target heights span 8–92% of the axis maximum\. The pool means are also similar: 21\.23–22\.83 for the raw value and 48\.84–49\.37% for normalized height\. Each of the four normalized\-height bins contains at least 18\.8% of every pool\. The selected pools retain all target category and series indices available in the generated 5,932\-query set\. Later indices are less frequent because they occur only in charts with more categories or series, a pattern already present in the generated pool\.

Figure 11:Composition of the evaluation pools\. \(a\) Cross\-model eligibility among feasible image pairs\. \(b\) Exact original target\-value frequencies\. \(c\) Target height normalized by the original axis maximum\. \(d–e\) Exact target category and series positions, including the complete generated pool as a reference\. \(f\) Realized numbers of categories \(CC\) and series \(SS\)\. Panels \(b\)–\(f\) count each bundle once rather than repeating the same target for every condition\.Original target valueNormalized target height \(%\)PoolMeanSDMinQ1MedianQ3–MaxMeanSDMinQ1MedianQ3–MaxAxis21\.2312\.542112030–5548\.8425\.388\.0025\.7148\.4570\.00–92\.00Visual22\.7912\.982122133–5549\.1023\.998\.0027\.8850\.0070\.00–92\.00Prompt22\.8312\.902122131–5549\.3724\.378\.0030\.0048\.7370\.28–92\.00

Table 14:Target\-value summaries\. Normalized height is the original target value divided by the original chart’s axis maximum\. Each visual or prompt bundle is counted once\.Target category index \(%\)Target series index \(%\)Pool012345012Generated \(5,932\)20\.020\.020\.020\.013\.36\.642\.038\.719\.3Axis \(500\)19\.222\.221\.219\.013\.45\.042\.236\.821\.0Visual \(500\)19\.616\.425\.222\.611\.05\.245\.839\.015\.2Prompt \(500\)19\.623\.619\.219\.413\.44\.838\.043\.418\.6Table 15:Exact target\-position percentages\. The generated\-pool row uses all 5,932 queries; the three evaluation rows count one representative target per pool\. Later indices are structurally available only in charts with enough categories or series\.All three selected pools cover four\-, five\-, and six\-category charts\. The visual and prompt bundles require at least two series, whereas the axis pool can include single\-series charts\.

Number of categoriesNumber of seriesPool456123Axis scale14516219314184302Visual bundle1691791520251249Prompt bundle1711691600239261Table 16:Realized chart structures in the three evaluation pools\.Target category positionTarget series positionPoolLeft/firstMiddleRight/lastFirstMiddleLastSingleAxis scale1771681551979919014Visual bundle149182169229891820Prompt bundle167165168190982120Table 17:Relative positions of the queried category and series\. Category positions are grouped by normalized horizontal location: below one third, from one third through two thirds, and above two thirds\. Two\-series charts contribute first and last series positions; three\-series charts additionally contribute a middle position\.For completeness, the target\-series counts can be separated by chart structure\. In the axis pool, the 14 one\-series cases all use index 0; the 184 two\-series cases split 99/85 between indices 0/1; and the 302 three\-series cases split 98/99/105\. The corresponding two\-/three\-series splits are 145/106 and 84/89/76 for the visual bundle, and 120/119 and 70/98/93 for the prompt bundle\. This confirms that the smaller aggregate count at series index 2 follows structural availability rather than omission of the final series when it exists\.

### C\.4Counterfactual directions and distances

Edit directions are close to balanced rather than forced to be exactly equal\. Bar height always changes by one tick interval\. Legend and series distances are measured in series\-index steps; x\-label and category distances are measured in category\-index steps\. Results are shown in Table[18](https://arxiv.org/html/2609.13745#A3.T18)

InterventionRealized distribution over 500 queriesAxis scalePair orientation original\-to\-scaled/scaled\-to\-original: 263/237\. Directed tick\-step transitions5→105\{\\rightarrow\}10,10→510\{\\rightarrow\}5,10→2010\{\\rightarrow\}20, and20→1020\{\\rightarrow\}10: 169, 157, 94, and 80\. The scale ratio is two for every pair\.Bar heightIncrease/decrease: 268/232\. Signed changes\+5,\+10,−5,−10\+5,\+10,\-5,\-10: 135, 133, 121, and 111\. Every edit changes the target by one tick interval\.Legend swapHigher/lower series index: 274/226\. Index distance 1/2: 422/78\.X\-label swapLeft/right: 243/257\. Index distances 1–5: 187, 165, 102, 39, and 7\.Prompt labelLeft/right: 259/241\. Index distances 1–5: 233, 134, 85, 31, and 17\.Prompt seriesHigher/lower series index: 244/256\. Index distance 1/2: 421/79\.Prompt label \+ seriesLabel left/right: 248/252; series higher/lower: 237/263\. Label distances 1–5: 201, 155, 93, 39, and 12\. Series distance 1/2: 413/87\.Table 18:Realized counterfactual directions and index distances\.
### C\.5ROI area audit

Table[19](https://arxiv.org/html/2609.13745#A3.T19)reports ROI area as a fraction of the rendered image\. This normalized quantity is comparable across input resolutions, unlike raw pixel counts\. The bar\-difference ROI is the smallest image region in both models, covering approximately 0\.6% of the image on average\.

InternVL3\.5 448Qwen2\.5\-VL 1344ROIMeanMedianMinMaxMeanMedianMinMaxAxis scale3\.0482\.7592\.3514\.3142\.3092\.0891\.7843\.274Bar union2\.4622\.3370\.7115\.2732\.1061\.9510\.5654\.677Bar difference0\.6670\.6480\.2931\.1300\.5810\.5630\.2451\.015Bar intersection1\.7951\.7520\.3034\.7171\.5241\.4750\.2194\.172Legend swap0\.9180\.9300\.6851\.0850\.6840\.7260\.5310\.776X\-label swap1\.3361\.3341\.3211\.3471\.1221\.1241\.1201\.124Table 19:ROI area as a percentage of the rendered image\.##### Audit summary\.

The synchronized evaluation set covers the full realized target\-value range, all chart structures permitted by each intervention pool, all target\-position groups, both edit directions, and multiple edit distances\. The principal within\-bundle results use paired queries, while source\-chart cluster and chart\-equal analyses account for repeated source charts\. These checks support the reported conclusions for jointly successful original–counterfactual cases; failure\-case mechanisms are outside the scope of this evaluation\.

Figure 12:Two\-layer residual\-window patching profiles\.Figure 13:Four\-layer residual\-window patching profiles\.Figure 14:Example of the bar\-height\-difference counterfactual\. The target bar value changes while the patch ROI covers only the changed height band\.Figure 15:Example of the bar\-height\-union counterfactual\. The target bar value changes while the patch ROI covers the geometric union of the original and counterfactual target bars\.Figure 16:Example of the bar\-height\-intersection counterfactual\. The target bar value changes while the patch ROI covers the geometric intersection of the original and counterfactual target bars\.Figure 17:Example of the legend\-swap counterfactual\. The semantic perturbation is induced by swapping legend items; the patch ROI covers the swapped legend entries\.Figure 18:Example of the x\-label\-swap counterfactual\. The semantic perturbation is induced by swapping two x\-axis labels; the patch ROI covers the swapped label slots\.Figure 19:Example of the axis\-scale counterfactual\. The y\-axis scale is rewritten while the patch ROI covers the affected y\-axis tick\-text region\.