VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback?

# VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback?
Source: [https://arxiv.org/html/2608.10408](https://arxiv.org/html/2608.10408)
Mizanur Rahman‡,Arshia Azimlu‡, Shadikur Rahman‡22footnotemark:2,Md Tahmid Rahman Laskar‡22footnotemark:2,Amran Bhuiyan‡22footnotemark:2, Shafiq Joty$,¶,Enamul Hoque Prince‡11footnotemark:1 ‡York University $Nanyang Technological University¶Salesforce AI Research

###### Abstract

Vision\-language models \(VLMs\) have shown strong capabilities in generating visualization code from textual or visual specifications\. However, real\-world visualization authoring is inherently iterative: users frequently revise existing visualizations to repair flawed charts or adapt them to desired styles\. Existing benchmarks primarily evaluate generation from scratch, leaving visualization code editing from multimodal feedback largely unexplored\. We introduceVisEditBench, a benchmark of 1,395 human\-annotated visualization code\-editing tasks grounded in realistic visualization workflows and failure cases\. VisEditBench covers two practical settings:feedback\-guided repair, where models revise visualization code using buggy or marked charts together with textual feedback, andreference\-guided restyling, where models modify code to match a target chart image\. Evaluating 20 state\-of\-the\-art VLMs reveals that visualization code editing remains challenging: Claude\-4\.6\-Sonnet achieves the best overall pass rate of 74\.46%, while most open\-source models remain below 50%\. Performance is particularly weak on visually grounded style adaptation, where Claude\-4\.6\-Sonnet achieves only 55\.71%\. To establish a strong baseline, we further proposeVisEditAgent, a render\-grounded editing framework that iteratively generates, executes, validates, and refines candidate edits\. Built on GPT\-4o, VisEditAgent improves overall pass rate from 55\.75% to 67\.99%, demonstrating the importance of render\-grounded feedback for faithful visualization editing\. We will release VisEditBench at[https://github\.com/vis\-nlp/VisEditBench](https://github.com/vis-nlp/VisEditBench)\.

\\undefine@key

newfloatplacement\\undefine@keynewfloatname\\undefine@keynewfloatfileext\\undefine@keynewfloatwithin

VisEditBench: Can Vision\-Language Models Edit Visualization Code from Multimodal Feedback?

Mizanur Rahman‡††thanks:Corresponding authors: \{mizanurr,enamulh\}@yorku\.ca, Arshia Azimlu‡††thanks:These authors contributed equally\.,Shadikur Rahman‡22footnotemark:2,Md Tahmid Rahman Laskar‡22footnotemark:2,Amran Bhuiyan‡22footnotemark:2,Shafiq Joty$,¶,Enamul Hoque Prince‡11footnotemark:1‡York University$Nanyang Technological University¶Salesforce AI Research

## 1Introduction

![Refer to caption](https://arxiv.org/html/2608.10408v1/x1.png)Figure 1:Overview of a VisEditBench task\. Given visualization code, visual feedback or a reference chart, and a language instruction, the model must generate revised executable code that repairs or restyles the visualization while preserving the underlying data semantics\.Data visualizations play a central role in modern data analysis and communication, enabling data scientists, business analysts, journalists, policymakers, and researchers to explore patterns, communicate insights, and support decision\-making across domainshoque2024natural;rahman2025llm\. To support increasingly complex visualization workflows, recent advances in vision\-language models \(VLMs\) have driven rapid progress in automated visualization, including chart generation from natural\-language instructionsrahman2025text2vis;maddigan2023chat2visand visualization code reconstruction from chart imagesyang2025chartmimic;wu2025plot2code\.

Yet real\-world visualization authoring rarely ends with the first generated chart\. Even when models produce executable visualizations, the outputs often fail to fully match user intent, visual preferences, accessibility requirements, or publication constraintsrahman2025text2vis;chen2024viseval\. As a result, users frequently inspect rendered charts, revise the underlying code, and rerender the output through iterative refinement\. In many workflows, charts must also be adapted to match preferred visual styles or reference designs, since default visualizations are often stylistically homogeneous and poorly aligned with communicative goals\. Figure[1](https://arxiv.org/html/2608.10408#S1.F1)illustrates a typical repair workflow: although the user already has visualization code and a rendered chart, the output remains visually flawed; labels overlap, annotations obscure important regions, and visual emphasis is misplaced\. Even seemingly minor edits can trigger cascading layout and styling changes, requiring repeated rounds of debugging and visual inspection\. This repair\-and\-restyling cycle is one of the most common and time\-consuming bottlenecks in practical visualization workflowsharper2017converting\. Unlike one\-shot generation, these workflows require localized edits that preserve data semantics, layout structure, encodings, and analytical meaning while modifying only the desired visual properties\. Visualization authoring is therefore fundamentally aniterative multimodal editing problem\. Solving this problem requires grounded multimodal reasoning\. A model must determinewhatis visually wrong,whichcode regions should change,howthe rendered chart should evolve, andwhichaspects of the original visualization must remain preserved\. This demands substantially richer reasoning than standard code generation or chart reconstruction\.

Despite rapid progress in VLMs, existing benchmarks largely overlook this setting\. Text\-to\-visualization benchmarks mainly evaluate chart generation from natural\-language queries or data tablesluo2021nvbench;liu2021advisor;rahman2025text2vis;chen2024viseval, while chart\-to\-code benchmarks evaluate code reconstruction from chart imagesyang2025chartmimic;wu2025plot2code\. Neither captures a common workflow where users already have visualization code and want models to iterativelyeditrendered visualizations using multimodal feedback or reference charts\. More general multimodal software\-engineering benchmarksyang2024swestudy visual bug fixing, but do not address visualization\-specific challenges such as repairing misleading encodings, resolving cluttered layouts, or matching reference chart styles while preserving data semantics\.

To address this gap, we introduceVisEditBench, a benchmark for evaluating visualization code editing from multimodal feedback\. Each task provides existing visualization code, a rendered chart, and a natural\-language editing instruction, and requires the model to generate revised executable code that produces the desired visualization\. VisEditBench supports two practical editing settings\. Infeedback\-guided repair, models receive buggy or human\-marked charts together with textual feedback and must repair the visualization while preserving its intended meaning \(Fig\.[1](https://arxiv.org/html/2608.10408#S1.F1)\)\. Inreference\-guided restyling, models receive a target chart image and must adapt the original visualization to match the reference style without altering the underlying data semantics\. Across these settings, VisEditBench contains 1,395 human\-annotated tasks grounded in realistic visualization issues collected from Stack Overflow, Matplotlib and Vega\-Lite issue reports, and diverse real\-world visualization datasets\.

Beyond executability, successful visualization editing requires edits that are visually faithful, semantically precise, and effective\. To support fine\-grained analysis, VisEditBench introduces a taxonomy of eight visualization\-editing intents spanning correctness repair, readability improvement, style adaptation, consistency harmonization, robustness improvement, structural transformation, constraint satisfaction, and style\-aware repair \(Fig\.[3](https://arxiv.org/html/2608.10408#S3.F3)\)\. Together, these settings capture realistic visualization workflows requiring visual reasoning, code understanding, and multimodal grounding\.

We evaluate 20 state\-of\-the\-art VLMs on VisEditBench in a zero\-shot setting using executability, task accuracy, readability and clarity, visual quality, visual similarity, and final pass rate\. Results show that visualization editing remains challenging: Claude\-4\.6\-Sonnet achieves the best overall pass rate at 74\.46%, while most open\-source models remain below 50%\. Performance is particularly weak on visually grounded style adaptation, where even Claude\-4\.6\-Sonnet achieves only 55\.71%\. To further study how rendering feedback can improve editing, we introduceVisEditAgent, a render\-grounded baseline, that generates multiple candidate revisions, executes and renders them, visually validates the outputs, and iteratively refines the selected solution\. Using GPT\-4o as the base model, VisEditAgent improves overall pass rate from 55\.75% to 67\.99%, demonstrating that render\-grounded feedback is critical for visually faithful visualization editing\.

In summary, our contributions include: \(i\)VisEditBench, a benchmark of 1,395 human\-annotated visualization code\-editing tasks covering feedback\-guided repair and reference\-guided restyling across diverse visualization\-editing intents; \(ii\) a structuredevaluation frameworkmeasuring executability, task accuracy, chart readability and clarity, visual quality, visual similarity, and final pass rate; \(iii\)zero\-shot evaluationsof 20 state\-of\-the\-art VLMs, revealing major limitations in executable and visually faithful chart editing; and \(iv\)VisEditAgent, a render\-grounded baseline that improves visualization editing through execution feedback, visual validation, and iterative refinement\.

## 2Related Work

Text\-to\-VisualizationExisting visualization benchmarks and systems have primarily studied how models generaterahman2025text2vis, understandhoque2022chartquestionansweringstate, or reconstruct chartswu2025plot2codefrom natural\-language queries, tabular data, chart images, and analytical intents \(Tab\.[6](https://arxiv.org/html/2608.10408#A1.T6)\)\. WikiSQLzhong2017seq2sqlsupports natural\-language\-to\-SQL parsing over tables, while nvBenchluo2021nvbenchextends this setting to large\-scale cross\-domain NL2VIS by synthesizing paired natural\-language queries and visualization specifications from NL\-to\-SQL benchmarks\. Later systems explore visualization recommendation, specification generation, and code generation from free\-form analytical queriesliu2021advisor;dibia2019data2vis;narechania2020nl4dv;song2022rgvisnet;maddigan2023chat2vis\. Recent LLM\-based benchmarks, including Text2Visrahman2025text2visand VisEvalchen2024viseval, evaluate whether large language models can generate visualizations across diverse real\-world datasets\. ChartBenchxu2023chartbenchfocuses on chart comprehension and visual reasoning, while ChartLlamahan2023chartllama, ChartMimicyang2025chartmimic, and Plot2Codewu2025plot2codetest chart reasoning or visualization\-code reconstruction from chart images\.

However, existing benchmarks do not evaluate iterative visualization editing from multimodal feedback, where users revise existing visualization code using rendered feedback or reference charts while preserving data semantics \(Tab\.[6](https://arxiv.org/html/2608.10408#A1.T6)\)\. VisEditBench addresses this gap by evaluating visualization code editing from multimodal feedback through feedback\-guided repair and reference\-guided restyling tasks\.

##### Multimodal Code Generation\.

Recent work has extended code generation to multimodal settings, where models generate or edit code from visual inputsli2024mmcode;si2025design2code;wu2025plot2code\. Early UI\-to\-code systems translated screenshots or sketches into executable interfacesbeltramelli2018pix2code;robinson2019sketch2code, while later work improved screenshot\-to\-code generation using vision\-code architectures and visual feedbacksoselia2023learning\. More recent benchmarks study visually grounded programming tasks, including diagram\-based programming \(MMCodeli2024mmcode\), screenshot\-to\-webpage generation \(Design2Codesi2025design2code\), SVG editing \(SVGEditBenchnishina2024svgeditbench\), and visual software bug fixing \(SWE\-bench Multimodalyang2024swe\)\. In contrast, VisEditBench focuses on editing and restyling existing visualization code from multimodal feedback while preserving data semantics\.

![Refer to caption](https://arxiv.org/html/2608.10408v1/emnlp2020-templates/imgs/FIG3_PDF21.png)Figure 2:Overview of the VisEditBench construction pipeline\. We collect real visualization issues and real\-world chart/data examples, construct multimodal editing tasks, cross\-review candidates for quality, and retain the final benchmark with metadata on editing intents, problem types, chart types, and visualization libraries\.

## 3VisEditBench

We introduce VisEditBench, a benchmark of 1,395 human\-annotated visualization code\-editing tasks from multimodal feedback\.

![Refer to caption](https://arxiv.org/html/2608.10408v1/x2.png)Figure 3:Examples from VisEditBench across eight editing intents\. Each task pairs input visualization code with a buggy, human\-marked, or reference chart image and a natural\-language instruction; the model must output revised executable code that renders the desired visualization\.### 3\.1Data Collection

We designed VisEditBench to capture realistic visualization editing workflows rather than synthetic chart\-generation prompts\. To ensure both realism and diversity, we combine two complementary sources: real user\-reported visualization issues and model\-generated chart failures on real\-world datasets\. Fig\.[2](https://arxiv.org/html/2608.10408#S2.F2)illustrates the collection process\.

First, we manually collected 120 real visualization\-editing cases from Stack Overflow and Matplotlib/Vega\-Lite GitHub issue discussions\. We retained only examples containing a user\-reported issue, visualization code, and a corresponding rendered buggy chart with clear visual problems requiring repair, improvement, or restyling\. These naturally occurring cases include cluttered layouts, unreadable labels, malformed axes, misleading encodings, annotation failures, broken legends, and rendering inconsistencies\.

Second, we use the Text2Vis benchmarkrahman2025text2vis, whose tables and queries are drawn from realistic data sources such as Statistastatista, Pew Researchpewresearch, Our World in Dataowid, and OECDoecd\. We reran both frontier and open\-source models on Text2Vis queries, executed the generated code, and manually inspected the rendered charts to identify visual, semantic, and presentation failures\. We retained chart\-code pairs only when the rendered output exhibited clear visualization errors or imperfect visual design, such as layout problems, incorrect encodings, unreadable labels, or poor presentation quality\. These retained cases therefore reflect realistic examples where current models fail in zero\-shot visualization generation\. This process substantially increases diversity beyond naturally reported bugs, covering not only correctness repair, but also style adaptation, consistency harmonization, robustness improvement, structural transformation, and style\-aware repair\.

### 3\.2Annotation and Quality Control

From these sources, four annotators created an initial pool of 1,648 candidate editing tasks\. For each task, annotators wrote a natural\-language editing instruction and assigned metadata including editing intent, problem type, visualization library, and chart type\. Each candidate was then independently cross\-reviewed by another annotator to verify that the task was visually grounded, realistic, clearly specified, and solvable through code edits\. A candidate was retained only when the reviewer judged that the code, chart image, instruction, and metadata together formed a valid editing task with an observable visual target and a feasible code\-level solution\.

To measure annotation reliability, we computed inter\-annotator agreement on the binary accept/reject decision across all candidates\. Annotators agreed on 84\.65% of cases, indicating strong consensus on what constitutes a valid visualization editing task\. We removed the remaining examples because they were ambiguous, insufficiently grounded in the image, duplicated, unrealistic, underspecified, or not clearly solvable through visualization code editing\. This filtering process resulted in the final VisEditBench dataset of 1,395 high\-quality tasks\. Each retained instance contains existing visualization code, a chart image, textual feedback or editing instruction, visual feedback or a reference chart, and structured metadata for fine\-grained analysis\.

Table 1:Statistics and diversity of VisEditBench\. Counts and percentages are computed over the final 1,395 examples\.Editing Intent \(Count / %\)Problem Type \(Count / %\)Correct\.RepairQualityImprov\.Robust\.Gen\.StyleAdapt\.Refactor/Transform\.ConsistencyHarmon\.ConstraintSatis\.Style\-awareRepairLayout &GeometryAnnotation &LabelingEncoding &MappingDataTransform\.Other529 / 37\.9446 / 32\.0116 / 8\.370 / 5\.063 / 4\.559 / 4\.252 / 3\.749 / 3\.5802 / 57\.5348 / 24\.9156 / 11\.263 / 4\.526 / 1\.9

Cause Type \(Count / %\)Complexity \(Count / %\)Chart Type \(Count / %\)Library \(Count / %\)Multi\-causeSingle\-causeEasyMediumHardBarLineBoxplotScatterOtherMatplotlibVega\-Lite797 / 57\.1598 / 42\.9516 / 37\.0498 / 35\.7381 / 27\.3679 / 48\.7522 / 37\.454 / 3\.952 / 3\.788 / 6\.31156 / 82\.9239 / 17\.1

### 3\.3Dataset Diversity

Table[1](https://arxiv.org/html/2608.10408#S3.T1)summarizes the diversity of VisEditBench\. The dataset spans eight editing intents, with Correctness Repair \(37\.9%\) and Quality Improvement \(32\.0%\) forming the largest categories, reflecting the prevalence of misleading, cluttered, or visually ineffective charts in real workflows\. VisEditBench also covers diverse problem types and difficulty levels\. Layout & Geometry issues are most common, followed by Annotation & Labeling, Encoding & Mapping, and Data Transformation\. These categories include failures such as overlapping labels, malformed axes, incorrect encodings, broken annotations, and data\-processing errors\. Importantly, 57\.1% of tasks are multi\-cause, requiring coordinated edits across multiple chart components rather than isolated fixes\. The dataset further spans varying levels of complexity, including a substantial number of challenging visually grounded editing problems \(27\.3% hard problems\)\. Difficulty labels were assigned using GPT\-5 under a fixed rubric and then manually reviewed and verified by human annotators \(classification prompt is in Figure[7](https://arxiv.org/html/2608.10408#A1.F7)\)\.

VisEditBench spans diverse chart types, editing behaviors, and visualization paradigms\. Bar and line charts are the most common chart families, reflecting their prevalence in practical visualization workflows, while the benchmark also includes scatter plots, boxplots, heatmaps, treemaps, histograms, waterfall charts, tables, and composite visualizations\. The benchmark further covers both imperative and declarative visualization paradigms through 1,156 Matplotlib examples and 239 Vega\-Lite examples\. Matplotlib tasks often require procedural edits involving layout control, annotation placement, and axis formatting, whereas Vega\-Lite tasks require modifying declarative specifications such as encodings, scales, legends, layers, and configuration fields\.

## 4Methodology

### 4\.1Task Formulation

We formulate visualization code editing as a multimodal code revision task\. Each example is defined asxi=\(ci,Ii,ui,ci′\)x\_\{i\}=\(c\_\{i\},I\_\{i\},u\_\{i\},c\_\{i\}^\{\\prime\}\), wherecic\_\{i\}is the input visualization code,IiI\_\{i\}is a rendered chart image,uiu\_\{i\}is a natural\-language editing instruction, andci′c\_\{i\}^\{\\prime\}is the revised visualization code\. Given\(ci,Ii,ui\)\(c\_\{i\},I\_\{i\},u\_\{i\}\), the model must generate executable codeci′c\_\{i\}^\{\\prime\}that produces the desired visualization while preserving the relevant data semantics\.

VisEditBench supports two practical editing settings\. Infeedback\-guided repair,IiI\_\{i\}contains a flawed or human\-marked chart, and the model must repair the visualization according to the provided feedback\. Inreference\-guided restyling,IiI\_\{i\}serves as a target chart or style reference, and the model must revise the original visualization to visually align with the reference while preserving its underlying analytical meaning\.

![Refer to caption](https://arxiv.org/html/2608.10408v1/x3.png)Figure 4:VisEditAgent framework\. Given visualization code, a buggy, human\-marked, or reference chart image, an instruction, and an edit intent, VisEditAgent plans the edit, generates candidate revisions, executes and renders them, validates the outputs, selects the best candidate, and refines it through iterative refinement rounds\.
### 4\.2Model Inference

We evaluate 20 state\-of\-the\-art VLMs spanning both closed\-source and open\-source model families\. Closed\-source models include multimodal systems from the GPT, Gemini, and Claude familiesopenai2024gpt4technicalreport;geminiteam2024gemini15unlockingmultimodal, while open\-source models cover recent VLM and code\-oriented families including Qwen, InternVL, LLaVA, and Gemmayang2024qwen2;jiang2023mistral;grattafiori2024llama;DBLP:journals/corr/abs\-2406\-11931;guo2025deepseek;roziere2023code\. This setup enables direct comparison between frontier proprietary systems and deployable open\-source alternatives under a unified visualization editing protocol\.We use a fixed decoding configuration for all models to support reproducibility\. Full inference and evaluation parameters are provided in App\.[A\.1](https://arxiv.org/html/2608.10408#A1.SS1)\(Tab\.[10](https://arxiv.org/html/2608.10408#A1.T10)\)\.

### 4\.3VisEditAgent

While zero\-shot prompting can produce plausible visualization edits, real\-world visualization authoring is inherently iterativemunzner2014visualization: authors repeatedly modify code, render charts, inspect the output, and refine the visualization through successive feedback loops\. This process is especially important for visually grounded edits involving layout, annotations, encodings, and style, where small code changes can trigger cascading visual errors or distort underlying data semantics\. Motivated by this workflow, we introduceVisEditAgent, a strong render\-grounded baseline for visualization code editing\. Rather than relying on a single\-pass response, VisEditAgent iteratively generates, renders, validates, and refines candidate edits using execution and visual feedback \(Fig\.[4](https://arxiv.org/html/2608.10408#S4.F4)\)\.

\(1\) Edit Planning\.Given the input code, chart image, instruction, and editing intent, the framework first identifies the likely visual issue, relevant code regions, and required modification strategy\. This stage grounds the edit in both the rendered visualization and the underlying code structure\.

\(2\) Candidate Generation\.VisEditAgent generates multiple candidate code edits representing different implementations of the requested modification\. Each candidate is then executed and rendered into a chart image, enabling the framework to jointly evaluate program\-level correctness and visualization\-level quality\. Exploring multiple rendered candidates is particularly important because many visualization edits admit several plausible yet visually different solutions\.

\(3\) Visual Validation and Selection\.Rendered candidates are evaluated using the task instruction, input chart, and reference or target image\. The framework assesses whether each candidate satisfies the requested edit, preserves the intended data semantics, and visually aligns with the desired output\. VisEditAgent then selects the strongest candidate based on execution success and visual alignment\.

\(4\) Feedback\-Based Refinement\.The selected candidate is further refined using execution diagnostics, rendering failures, and visual\-validation feedback\. This refinement stage helps correct common one\-shot failure modes\.

Table 2:Pass rate by editing intent on VisEditBench\. All values are percentages \(%\)\. Overall averages all examples\.ModelCorrectnessRepairQualityImprovementRobustness /GeneralizationStyleAdaptationConstraintSatisfactionConsistencyHarmonizationRefactor /TransformationStyle\-AwareError RepairOverallClosed\-Source ModelsGPT\-4o56\.0867\.3457\.7610\.0063\.4647\.4642\.8630\.6155\.75GPT\-5\-Mini55\.5164\.6471\.5547\.1467\.3055\.9358\.7351\.0260\.00GPT\-561\.6075\.4556\.8957\.1459\.6166\.1069\.8451\.0265\.68Gemini\-3\.0\-Flash17\.9619\.289\.4835\.715\.7715\.259\.520\.0017\.06Claude\-4\.5\-Sonnet60\.8365\.3475\.0052\.8667\.3142\.3874\.6051\.0262\.73Claude\-4\.6\-Sonnet70\.3479\.7386\.2055\.7180\.7664\.4085\.7161\.2274\.46Open\-Source ModelsGemma\-3\-4B30\.7925\.001\.000\.003\.8516\.953\.181\.0021\.29Gemma\-3\-12B42\.9747\.976\.891\.4313\.4628\.817\.9410\.2034\.96Gemma\-3\-27B49\.2456\.7623\.280\.0026\.9242\.3712\.7012\.2542\.95InternVL\-3\.5\-1B11\.417\.880\.000\.001\.928\.471\.592\.047\.41InternVL\-3\.5\-2B25\.6721\.850\.000\.003\.8520\.340\.006\.1218\.13InternVL\-3\.5\-4B34\.0333\.788\.620\.009\.6128\.811\.582\.0426\.33InternVL\-3\.5\-8B42\.9641\.227\.750\.0011\.5432\.2012\.7010\.2033\.17InternVL\-3\.5\-14B41\.6549\.783\.450\.003\.8537\.297\.9410\.2134\.83Pixtral\-12B44\.6847\.075\.170\.0015\.3830\.516\.3510\.2035\.32Qwen3\-VL\-2B24\.5220\.491\.000\.005\.7716\.957\.946\.1217\.55Qwen3\-VL\-4B51\.1150\.901\.721\.427\.6937\.284\.7616\.3238\.84Qwen3\-VL\-8B49\.0452\.0228\.440\.005\.7052\.4426\.9814\.2942\.09Qwen3\-VL\-30B\-A3B49\.8053\.3844\.8315\.7138\.4642\.3747\.6120\.4147\.05Qwen3\-VL\-32B54\.9559\.0050\.0021\.4248\.1054\.2336\.5116\.3251\.72

## 5Evaluation

### 5\.1Evaluation Criteria

Evaluating visualization editing requires measuring not only code executability, but also whether the edited chart satisfies the intended visual revision while remaining readable, visually coherent, and faithful to the target output\. We therefore evaluate each output using five criteria: \(i\)Code Execution, which checks whether the generated code executes successfully and renders a valid chart; non\-executable, timed\-out, or empty outputs receive zero scores for all remaining metrics; \(ii\)Task Accuracy, which measures whether the edited chart satisfies the requested instruction or matches the target/reference chart while preserving the intended visualization semantics; \(iii\)Readability and Clarity, which evaluates the legibility and organization of labels, legends, axes, ticks, annotations, and other chart elements; \(iv\)Visual Quality, which assesses overall presentation quality, including layout, scaling, color usage, chart appropriateness, and visual polish; and \(v\)Visual Similarity, which measures how closely the rendered output aligns with the corrected or reference chart image\.

Automatic Evaluator\.We use a rubric\-based VLM evaluator with fixed prompts, deterministic settings, explicit scoring criteria, and a fixed input order\. The evaluator receives the task instruction, the input or reference chart, and the rendered model output, without access to model identities\. By default, GPT\-4o serves as the evaluator for all visual metrics; when evaluating GPT\-4o outputs, we instead use Gemini 2\.5 Pro to avoid self\-evaluation bias\. We further validate the evaluator through human assessment using the same rubric and report agreement between automatic and human judgments\.

Scoring and Final Pass Rate\.Task accuracy, readability and clarity, and visual quality are scored on a 0–5 scale with 0\.5\-point increments, while visual similarity is scored from 0–100\. We additionally report a strict final pass rate: an example passes only if the code executes successfully, task accuracy is at least 4\.5, readability and clarity and visual quality are at least 4\.0, and visual similarity is at least 90\. These thresholds intentionally measure complete task success rather than partial improvement, ensuring that edited charts are executable, visually coherent, instruction\-faithful, and aligned with the intended target\. The full evaluation rubric, including score definitions and intent\-specific visual similarity rules, is provided in Table[9](https://arxiv.org/html/2608.10408#A1.T9)\.

### 5\.2Main Results

#### 5\.2\.1Zero\-Shot Performance

Tables[2](https://arxiv.org/html/2608.10408#S4.T2)and[3](https://arxiv.org/html/2608.10408#S5.T3)reveal a substantial gap between generating executable visualization code and performing visually faithful visualization editing\. Overall, closed\-source models substantially outperform open\-source models, but even the strongest model remains far from solving visualization code editing\. Claude\-4\.6\-Sonnet achieves the best overall pass rate at 74\.46%, followed by GPT\-5 at 65\.68% and Claude\-4\.5\-Sonnet at 62\.73%\. Among open\-source models, Qwen3\-VL\-32B performs best with 51\.72%, while most open\-source models remain below 50%\. Smaller models perform much worse, such as InternVL\-3\.5\-1B at 7\.41% and Qwen3\-VL\-2B at 17\.55%, suggesting that visualization editing requires strong multimodal and reasoning ability\.

Table 3:Metric\-level scores on VisEditBench\. Executability is reported as a percentage, Visual Similarity on a 0–100 scale, and other metrics on a 5\-point scale\.ModelCodeExec\.TaskAcc\.Read\. &ClarityVisualQualityVisualSimilarityClosed\-Source ModelsGPT\-4o93\.314\.044\.033\.9578\.13GPT\-5\-Mini88\.204\.114\.124\.0780\.65GPT\-594\.104\.134\.104\.0984\.13Gemini\-3\.0\-Flash46\.021\.832\.072\.0442\.85Claude\-4\.5\-Sonnet94\.464\.414\.384\.3886\.52Claude\-4\.6\-Sonnet96\.194\.574\.544\.5589\.42Open\-Source ModelsGemma\-3\-4B82\.302\.703\.303\.3364\.78Gemma\-3\-12B87\.193\.573\.753\.7874\.20Gemma\-3\-27B88\.273\.923\.973\.9877\.34InternVL\-3\.5\-1B72\.811\.362\.622\.6553\.74InternVL\-3\.5\-2B80\.002\.393\.093\.1361\.47InternVL\-3\.5\-4B78\.713\.013\.283\.3165\.35InternVL\-3\.5\-8B83\.383\.473\.643\.6571\.25InternVL\-3\.5\-14B78\.953\.343\.463\.4868\.12Pixtral\-12B86\.043\.463\.663\.6972\.43Qwen3\-VL\-2B79\.932\.243\.093\.1362\.48Qwen3\-VL\-4B79\.713\.083\.393\.4367\.86Qwen3\-VL\-8B85\.833\.683\.793\.8274\.77Qwen3\-VL\-30B\-A3B89\.713\.924\.034\.0579\.19Qwen3\-VL\-32B87\.774\.054\.014\.0478\.38

A key finding is that executability alone is not the primary bottleneck for strong models\. Claude\-4\.6\-Sonnet and GPT\-4o execute successfully on 96\.19% and 93\.31% of examples, respectively, yet achieve substantially lower visual similarity scores of 89\.42 and 78\.13\. This indicates that models can often generate runnable code while still failing to preserve visual semantics, layout fidelity, or stylistic alignment with the target edit\.

Table 4:Pass rate by editing intent for zero\-shot models, VisEditAgent, and ablations\. All values are percentages \(%\)\.Model / MethodCorrect\.RepairQualityImprove\.Robust\.Gen\.StyleAdapt\.ConstraintSatis\.ConsistencyHarmon\.Refactor/Transform\.Style\-awareRepairOverallGPT\-4o Zero\-shot56\.0867\.3457\.7610\.0063\.4647\.4642\.8630\.6155\.75GPT\-4o \+ VisEditAgent64\.6478\.3761\.2262\.8565\.3864\.4160\.3244\.8967\.99Qwen3\-VL\-4B Zero\-shot51\.1150\.901\.721\.427\.6937\.284\.7616\.3238\.84Qwen3\-VL\-4B \+ VisEditAgent63\.1463\.0042\.2435\.7144\.2354\.2342\.8518\.3644\.98Qwen3\-VL\-8B Zero\-shot49\.0452\.0228\.440\.005\.7052\.4426\.9814\.2942\.09Qwen3\-VL\-8B \+ VisEditAgent71\.0867\.711\.8024\.2913\.4649\.1517\.4616\.3654\.53Ablations on GPT\-4o \+ VisEditAgentw/o Multi\-candidate Generation61\.4474\.4456\.0328\.5759\.6255\.9352\.3838\.7861\.94w/o Refinement58\.7971\.5252\.5941\.4357\.6954\.2449\.2134\.7059\.86

Performance also varies sharply across editing intents\. Stronger models perform relatively well on correctness repair and quality improvement, where edits often involve localized readability or layout fixes\. For example, Claude\-4\.6\-Sonnet reaches 70\.34% on correctness repair and 79\.73% on quality improvement, while GPT\-4o reaches 56\.08% and 67\.34%, respectively\. In contrast, visually grounded and coordinated edits remain difficult: Claude\-4\.6\-Sonnet achieves only 55\.71% on style adaptation, and GPT\-4o only 10\.00%\. Overall, VisEditBench requires more than code generation or chart reconstruction: successful models must jointly reason over code, visual feedback, user intent, and rendered chart quality\.

#### 5\.2\.2VisEditAgent Results

Table[4](https://arxiv.org/html/2608.10408#S5.T4)shows that VisEditAgent consistently improves over direct zero\-shot prompting\. Using GPT\-4o as the base model, VisEditAgent increases overall pass rate from 55\.75% to 67\.99%, demonstrating the importance of render\-grounded iterative refinement for visualization editing\. The largest gains appear on visually grounded editing tasks\. Style adaptation improves from 10\.00% to 62\.85%, consistency harmonization from 47\.46% to 64\.41%, and refactor/transformation from 42\.86% to 60\.32%\. Similar trends hold for open\-source models: for example, Qwen3\-VL\-4B improves from 1\.42% to 35\.71% on style adaptation and from 4\.76% to 42\.85% on refactor/transformation\. Table[5](https://arxiv.org/html/2608.10408#A1.T5)further shows consistent improvements in task accuracy, readability, visual quality, and visual similarity\. These results suggest that a central failure mode of zero\-shot VLMs is not merely code generation, but the inability to iteratively validate and refine edits against rendered visual feedback\. Multi\-candidate generation, rendering, visual validation, and refinement substantially improve the ability of models to produce edits that are both executable and visually faithful \(see Figure[5](https://arxiv.org/html/2608.10408#A1.F5)\)\.

To validate the automatic evaluation, we manually evaluated outputs from GPT\-4o zero\-shot and GPT\-4o \+ VisEditAgent on all 1,395 VisEditBench samples, resulting in 2,790 total model outputs\. We additionally evaluated Qwen3\-VL\-4B zero\-shot and Qwen3\-VL\-4B \+ VisEditAgent on a 500\-example stratified sample\. Five annotators assessed each output using the same rubric as the automatic evaluator, covering task accuracy, readability and clarity, visual quality, visual similarity, and final pass/fail judgment\. As shown in Table[7](https://arxiv.org/html/2608.10408#A1.T7), human evaluation confirms that VisEditAgent improves pass rate for both GPT\-4o and Qwen3\-VL\-4B\. The human scores also show strong agreement with the automatic evaluator, with Pearson correlations ranging from 83\.38 to 87\.00 across the four scalar metrics: task accuracy, readability and clarity, visual quality, and visual similarity \(Table[8](https://arxiv.org/html/2608.10408#A1.T8)\)\. This suggests that the automatic evaluation is well aligned with human judgments while enabling scalable assessment of visualization code editing performance\.

### 5\.3Ablation Studies

As shown in Tab\.[4](https://arxiv.org/html/2608.10408#S5.T4), removing multi\-candidate generation lowers VisEditAgent’s overall pass rate with GPT\-4o from 67\.99% to 61\.94%, while removing refinement lowers it to 59\.86%\. The largest drop occurs for style adaptation, where performance decreases from 62\.85% to 28\.57% and 41\.43%, respectively\. These results show that both candidate selection and feedback\-based refinement are important for visually faithful editing\.

### 5\.4Error Analysis

We conducted a qualitative error analysis on 500 randomly selected model outputs\. We identify four recurring failure modes, illustrated in Fig\.[6](https://arxiv.org/html/2608.10408#A1.F6)\.

Executable but visually incorrect outputs\.The code executes and renders a chart, but the edit is not satisfied\. Typical failures include unresolved label overlap, misplaced legends, cluttered annotations, and poor readability\.

Weak grounding in visual feedback\.Models fail to localize edits from marked or reference images\. They move annotations, legends, or labels to incorrect regions, or keep arrows pointing to the wrong data points\.

Poor reference\-style matching\.In style adaptation, models often make generic visual changes rather than preserving the reference layout, colors, fonts, legends, backgrounds, or annotation style\.

Incorrect or incomplete transformations\.For refactor/transformation tasks, models change the wrong chart type, omit required visual elements, alter encodings, or fail to preserve data semantics\.

## 6Conclusion

We introduceVisEditBench, the first benchmark for visualization code editing from multimodal feedback, moving beyond the one\-shot chart\-generation focus of prior work\. Across 1,395 human\-annotated tasks spanning feedback\-guided repair and reference\-guided restyling, our evaluation reveals a substantial gap between generating executable charts and performing visually faithful visualization editing, particularly for visually grounded and coordinated edits requiring layout, style, and semantic preservation\. As a strong baseline, we includeVisEditAgent, a render\-grounded framework that iteratively generates, executes, validates, and refines candidate edits using visual feedback\. This framework substantially improves editing performance, demonstrating that reliable visualization editing requires iterative multimodal reasoning rather than single\-pass code generation\. Together, VisEditBench and VisEditAgent establish a strong foundation for advancing visually grounded, feedback\-aware, and user\-aligned visualization authoring systems\.

## Limitations

VisEditBench currently covers two visualization libraries, Matplotlib and Vega\-Lite\. These libraries represent both imperative and declarative visualization paradigms and support many common chart\-editing scenarios\. However, the benchmark does not yet directly evaluate other widely used visualization ecosystems such as Plotly, D3\.js, and ggplot2\. Extending VisEditBench to these libraries would allow future work to study whether the observed model behaviors generalize across a broader range of visualization programming environments\.

Although VisEditBench includes diverse chart types, editing intents, problem types, and difficulty levels, it is not intended to exhaustively cover every possible visualization\-editing need\. The benchmark emphasizes common editing scenarios such as layout adjustment, label and annotation repair, encoding correction, style adaptation, and reference\-guided restyling\. More specialized domains may involve additional constraints, domain conventions, or visualization types that are not fully represented in the current dataset\.

## Ethical Considerations

VisEditBench is designed to support transparent and reproducible research on multimodal visualization code editing\. The benchmark is constructed from publicly available visualization resources, including user\-reported visualization issues and real\-world chart/data examples, and is used only for research and evaluation purposes\. During dataset construction, annotators retained only examples with concrete visualization code, clearly grounded visual feedback, and implementable editing goals\. We used LLM assistance only for language editing and polishing, while all technical content, dataset construction, experiments, and analyses were reviewed and verified by the authors\.

To improve data quality and reduce ambiguity, each candidate task was cross\-reviewed before inclusion in the final benchmark\. We removed examples that were underspecified, duplicated, unrealistic, insufficiently grounded in the chart image, or not clearly solvable through visualization code editing\. The benchmark does not aim to collect or evaluate sensitive personal information; its focus is on visualization code, rendered charts, editing instructions, and structured metadata\.

We maintained fairness in model comparisons by applying the same prompting protocol, execution environment, evaluation criteria, and scoring rubrics across open\-source and closed\-source models\. For automatic evaluation, we used a fixed rubric and avoided self\-evaluation bias by using a different evaluator when assessing outputs from the default evaluator model\. We also report limitations and failure modes to help prevent overclaiming model capabilities\. Overall, VisEditBench is intended to encourage reliable, visually grounded, and user\-aligned visualization authoring systems\. Finally, we used AI\-based writing assistants only to improve the presentation of the paper\.

## Acknowledgements

This research was supported by the Natural Sciences and Engineering Research Council \(NSERC\), Canada, Canada Foundation for Innovation, Compute Canada, and the CIRC grant on Inclusive and Accessible Data Visualizations and Analytics\.

## References

## Appendix AAppendices

Table 5:Metric\-level comparison of zero\-shot models and VisEditAgent\. Executability is reported as a percentage \(%\)\. Task Accuracy, Chart Readability and Clarity, and Visual Quality are reported on a 5\-point scale\. Visual Similarity is reported on a 0–100 scale\. Best results in each column are shown in bold\.Model / MethodCodeExec\.TaskAcc\.Read\. &ClarityVisualQualityVisualSimilarityGPT\-4o Zero\-shot93\.314\.044\.033\.9578\.13GPT\-4o \+ VisEditAgent96\.834\.374\.354\.3686\.30Qwen3\-VL\-4B Zero\-shot79\.713\.083\.393\.4367\.86Qwen3\-VL\-4B \+ VisEditAgent91\.933\.703\.923\.9477\.30Qwen3\-VL\-8B \+ VisEditAgent93\.454\.034\.134\.1676\.50

BenchmarkMain TaskInputCode GivenHuman FeedbackCode EditingRef\. RestylingEdit TaxonomyText2Visrahman2025text2visText\-to\-vis generationText \+ data△\\triangle✗✗✗✗VisEvalchen2024visevalNL2VIS evaluationText \+ data✗✗✗✗✗ChartMimicyang2025chartmimicChart\-to\-code generationChart image✗✗✗✗✗Plot2Codewu2025plot2codePlot\-to\-code generationPlot image△\\triangle✗✗✗✗SWE\-bench Multimodalyang2024sweVisual software repairIssue \+ repo\.✓✗✗✗✗VisEditBenchVisualization repair & restylingCode \+ multimodal feedback✓✓✓✓✓Table 6:Comparison of VisEditBench with related visualization and multimodal software benchmarks\. ✓ = yes, ✗ = no, and△\\triangle= partial\. VisEditBench uniquely evaluates editing existing visualization code from multimodal feedback, including feedback\-guided repair and reference\-guided restyling\.Table 7:Human evaluation results for GPT\-4o and Qwen3\-VL\-4B under zero\-shot and VisEditAgent settings\. GPT\-4o is evaluated on all 1,395 examples, while Qwen3\-VL\-4B is evaluated on a 500\-example stratified sample\. Code execution and pass rate are reported as percentages\. Task accuracy, readability and clarity, and visual quality use a 0–5 scale; visual similarity uses a 0–100 scale\.Model / MethodGPT\-4o Zero\-shot95\.004\.134\.104\.1482\.7051\.70GPT\-4o \+ VisEditAgent97\.004\.364\.414\.4089\.6059\.10Qwen3\-VL\-4B Zero\-shot80\.543\.113\.413\.4468\.1338\.91Qwen3\-VL\-4B \+ VisEditAgent91\.633\.683\.853\.9577\.2045\.45Table 8:Human–automatic evaluation agreement across all human\-evaluated outputs\. Pearson correlations are reported for scalar metrics; agreement is reported for the final pass/fail decision\.MetricPearson CorrelationTask Accuracy83\.38Readability & Clarity87\.00Visual Quality86\.50Visual Similarity86\.00Final Pass/Fail Agreement81\.00%![Refer to caption](https://arxiv.org/html/2608.10408v1/x4.png)Figure 5:Qualitative example comparing GPT\-4o zero\-shot with GPT\-4o \+ VisEditAgent\. The original chart contains crowded x\-axis labels and poorly positioned cumulative\-percentage annotations\. The editing instruction asks the model to improve label/annotation readability and spacing while preserving the chart semantics\. Although GPT\-4o zero\-shot produces executable code, it only partially addresses these issues, whereas VisEditAgent produces a cleaner chart with better\-positioned annotations and more readable labels\.![Refer to caption](https://arxiv.org/html/2608.10408v1/x5.png)Figure 6:Error analysis examples showing that models often produce executable and visually plausible charts while missing the intended edit\. \(a\) GPT\-4o reduces clutter but the chart remains unreadable due to overlapping 1K–256M request labels\. \(b\) GPT\-4o fails to move the legend outside the line chart, leaving it over the plotted data\. \(c\) InternVL\-14B produces a clean chart but does not match the dark reference style\. \(d\) Qwen3\-VL\-4B fails to transform the output into the reference visualization\.MetricRubricAllowed ScoresFor 0–5 metrics, the evaluator may use only the following scores: 0, 0\.5, 1, 1\.5, 2, 2\.5, 3, 3\.5, 4, 4\.5, and 5\. Visual similarity is scored from 0 to 100\.Task AccuracyScore range:0–5, in 0\.5\-point increments\.Correctness Repair and Quality Improvement:5 = fully fixes the issue shown in the image and instruction, preserves data meaning, and introduces no meaningful new problems; 4\.5 = almost fully fixes the issue, with only tiny remaining issues; 4\.0 = mostly fixes the issue, with minor remaining problems; 0 = invalid, irrelevant, unreadable, or does not address the requested fix at all\.All other intents:5 = closely matches the reference/target image and instruction, preserving data meaning unless the instruction asks otherwise; 4\.5 = almost closely matches the reference/target, with only very minor differences; 4\.0 = mostly matches the reference/target, with minor differences; 0 = invalid, irrelevant, unreadable, or does not match the reference/instruction at all\.Readability & ClarityScore range:0–5, in 0\.5\-point increments\.5 = labels, titles, axes, ticks, legends, fonts, and annotations are clear, readable, well\-positioned, and non\-overlapping\.0 = no valid readable chart, or chart elements are unreadable, clipped, or overlapped so badly that the chart cannot be interpreted\.Visual QualityScore range:0–5, in 0\.5\-point increments\.5 = polished, professional, visually clean, accessible, uses appropriate colors/scales/chart type, and communicates the intended information clearly\.0 = no valid visualization, irrelevant image, misleading visual design, or extremely poor visual quality\.Visual SimilarityScore range:0–100\.Correctness Repair and Quality Improvement:Compare the rendered chart to the clean corrected version of the original buggy/marked chart\. The rendered chart should remain visually similar to the original chart, but should fix the issue described in the instruction\. Do not penalize the rendered chart for not reproducing handwritten marks or the original bug\.Style Adaptation:Compare style similarity: colors, fonts, background, gridlines, mark style, layout, and overall appearance\. Data values may differ if the task uses the base data, but the style should match the reference image closely\.Style\-Aware Error Repair:The reference image is the target chart style/output and may show marked potential errors\. The rendered chart should recreate the clean target chart while avoiding the marked or anticipated issues\. Do not reward reproducing handwritten marks or buggy artifacts\.All other intents:The reference image is the target chart\. The rendered chart should match it as closely as possible in chart type, data, encodings, colors, labels, title, axes, legend, annotations, layout, and overall appearance\.Visual Similarity Scale100 = near\-perfect match under the intent\-specific rules; 90–99 = almost identical, only tiny differences; 80–89 = good match but minor differences; 60–79 = partially correct but noticeable differences; 40–59 = weak match; 1–39 = barely related; 0 = completely different, invalid, or no usable chart\.Intermediate Guide for 0–5 Metrics4\.5 = very good; only tiny issues\. 4\.0 = good; small issues but clearly successful\. 3\.5 = decent; mostly usable but with noticeable issues\. 3\.0 = average; important issues affect clarity/correctness\. 2\.5 = below average; several issues may mislead or confuse\. 2\.0 = poor; significant issues\. 1\.5 = very poor; major problems\. 1\.0 = barely usable or mostly wrong\. 0\.5 = almost completely failed but has a tiny relevant element\. 0\.0 = failed\.Evaluator RulesIf executable = 0, all scores must be 0\. The evaluator is instructed to be strict\. Handwritten annotations, circles, arrows, scribbles, highlights, and notes are ignored unless they describe what should be fixed\. A nice\-looking chart is not rewarded if it fails the instruction\. A chart that changes data meaning is not rewarded unless requested\.Table 9:Evaluation rubric used for scoring rendered visualization outputs\. Task accuracy, readability and clarity, and visual quality are scored on a 0–5 scale in 0\.5\-point increments\. Visual similarity is scored on a 0–100 scale\. The rubric is intent\-aware and distinguishes repair, style adaptation, style\-aware error repair, and other reference\-guided target\-matching tasks\.Difficulty Classification PromptYou are an expert annotator for VisEditBench, a benchmark for visualization code editing from multimodal feedback\.Your task is to classify one visualization editing sample asEasy,Medium, orHard\.You will be given:1\.The input visualization code\.2\.The input chart image, which may show a buggy chart, a human\-marked chart, or a reference/target chart\.3\.The natural\-language editing instruction\.4\.Metadata such as editing intent, problem type, chart type, and visualization library, if available\.Classify the task difficulty based on the expected effort required to produce the correct edited visualization code while preserving the intended data semantics\. Use the number of distinct issues as the main signal, while also considering visual reasoning, code reasoning, issue interaction, scope of required edits, reference\-style matching complexity, and risk of changing the underlying data meaning\.Easy\.A task is Easy if it contains one main issue and the required edit is local, direct, and clearly specified\. The issue is usually visible from the chart and can be fixed by modifying one chart property or a small number of nearby parameters\. The correct code location is usually obvious, and there is little risk of changing the underlying data semantics\.Typical Easy tasks include:•increasing font size or rotating tick labels;•moving a legend, title, or axis label;•fixing one simple label overlap;•changing a single color, marker, line style, or font property;•adjusting figure size, margins, or axis\-label spacing;•adding or removing one simple annotation\.Choose Easy when the task mostly requires fixing one localized issue with minimal visual\-code reasoning\.Medium\.A task is Medium if it contains two distinct issues, or if one issue requires coordinated changes across multiple chart components\. The task may require moderate reasoning over the image, instruction, and code, but the required edits are still reasonably well\-scoped\. Medium tasks do not need to be structurally complex; they only need to require more than one localized change or moderate visual\-code alignment\.Typical Medium tasks include:•fixing two visible issues in the same chart;•fixing multiple overlapping labels, ticks, or annotations;•adjusting axes, legends, titles, layout, and spacing together;•changing encodings while preserving the chart’s meaning;•improving readability across several chart components;•matching a reference style with several visible properties such as colors, fonts, gridlines, and layout;•modifying grouped, stacked, or multi\-series charts\.Choose Medium when the task requires two fixes, several coordinated edits, or moderate visual\-code reasoning, but does not require major structural transformation or highly complex semantic reasoning\.Hard\.A task is Hard if it contains three or more distinct issues, multiple interacting causes, structural code changes, complex reference matching, or careful preservation of data semantics while making nontrivial visual modifications\. A task can be Hard even if the instruction is short, when the correct solution requires several dependent edits or substantial reasoning about both the rendered image and the code\.Typical Hard tasks include:•fixing three or more visible issues in the same chart;•converting between chart types while preserving analytical meaning;•repairing a chart with multiple interacting errors;•matching a complex reference style involving layout, colors, fonts, annotations, legends, background, and mark design;•modifying faceted, layered, composite, or multi\-panel visualizations;•fixing issues caused by both data transformation and visual encoding;•coordinating changes across scales, legends, annotations, encodings, layout, and data processing\.Choose Hard when the task requires three or more fixes, interacting fixes, structural transformation, complex reference matching, or high risk of changing data semantics\.Issue\-count guideline\.Use the number of distinct issues as the main difficulty signal, while still considering reasoning complexity\. One localized issue usually suggests Easy, two distinct issues usually suggest Medium, and three or more distinct issues usually suggest Hard\. However, override this rule when complexity clearly changes the difficulty: a single issue can be Medium or Hard if it requires structural code changes, complex reference matching, nontrivial data transformation, or high risk of changing data semantics; multiple issues can be lower difficulty if they are trivial, independent, and solved by the same localized edit\.Decision rules\.Focus on the difficulty of producing the correct edited code, not merely understanding the instruction\. Do not label a task Hard only because the chart looks visually busy, and do not label a task Easy only because the instruction is short……Return only valid JSON:\{
"difficulty": "Easy" \| "Medium" \| "Hard",
"explanation": "One or two sentences explaining the label\."
\}Figure 7:Prompt used for difficulty classification\. GPT\-5 was used to classify each VisEditBench example as Easy, Medium, or Hard under this rubric\.### A\.1Inference and Evaluation Parameters

To support reproducibility, we report the decoding, rendering, and evaluation parameters used in our experiments in Table[10](https://arxiv.org/html/2608.10408#A1.T10)\.

ComponentConfigurationInference parametersFor direct model inference, each model receives the visualization image and the corresponding editing prompt as multimodal input\. The model is instructed to return only complete revised code, without markdown or explanation\. Unless otherwise stated, we use image detail =high, temperature = 0\.2, top\-p = 1\.0, maximum output tokens = 4096, and output format = complete executable visualization code only\.Evaluation parametersGenerated code is cleaned, executed, and rendered before scoring\. Vega\-Lite specifications are rendered usingvl\-convert\-python, while Python/Matplotlib outputs are executed using a non\-interactiveAggbackend\. Non\-executable outputs are assigned executability 0 and zero scores for all remaining metrics\. GPT\-4o is used as the default evaluator; for GPT\-4o outputs, Gemini 2\.5 Pro is used to avoid self\-evaluation bias\. The evaluator uses temperature = 0\.0, maximum output tokens = 1200, and image detail =high\. Task accuracy, readability and clarity, and visual quality are scored on a 0–5 scale in 0\.5\-point increments; visual similarity is scored on a 0–100 scale\.Final pass criteriaAn output is counted as a pass only if all conditions are satisfied:executable = 1, task accuracy≥4\.5\\geq 4\.5, readability and clarity≥4\.0\\geq 4\.0, visual quality≥4\.0\\geq 4\.0, and visual similarity≥90\\geq 90\.Table 10:Inference and evaluation parameters used in VisEditBench experiments\. These settings are fixed across model evaluations unless otherwise stated\.