ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats

Hugging Face Daily Papers Papers
chart-parsing benchmark vision-language evaluation multilingual open-source human-ai-annotation

Summary

ChartArena is a comprehensive bilingual benchmark for chart parsing that evaluates models across eight chart families and three visual scenarios (digital, printed, hand-drawn), using a human-agent annotation pipeline and format-agnostic evaluation. Evaluations of 26 MLLMs reveal that while proprietary models lead overall, open-source models are catching up, and diagrammatic structures and hand-drawn scenarios remain challenging.

Charts are a primary medium for conveying quantitative and relational information, yet systematically evaluating chart parsing models remains difficult. Existing benchmarks focus on narrow chart types and leave diagrammatic structures such as flowcharts and mind maps largely unaddressed, while models produce outputs in incompatible formats, and datasets rarely include the printed or hand-drawn images encountered in practice. To address these issues, we introduce ChartArena, a comprehensive bilingual benchmark covering eight chart families spanning both numeric charts and diagrammatic structures, each evaluated across three visual scenarios: digital renderings, printed photos, and hand-drawn photos. The dataset is built via a human-agent collaborative annotation pipeline with multi-stage human verification to ensure annotation reliability. To enable fair cross-model comparison, we further design a format-agnostic evaluation protocol that maps heterogeneous outputs into two canonical semantic spaces, a normalized triple view and a directed graph view, and scores them with structure-aware metrics. Through extensive evaluation of 26 leading MLLMs, we observe three consistent findings: (i) frontier proprietary models such as Gemini 3.1 Pro lead overall, yet the strongest open-source systems are rapidly closing the gap; (ii) document parsing models handle numeric charts reasonably but fall sharply behind on diagrammatic structures; and (iii) expert chart parsers remain limited to narrow chart families. Across all models, radar charts and hand-drawn scenarios stay especially challenging. These findings show that ChartArena exposes clear capability gaps and provides a unified foundation for future progress. ChartArena is publicly available at https://github.com/pspdada/ChartArena.
Original Article
View Cached Full Text

Cached at: 06/02/26, 03:36 PM

Paper page - ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats

Source: https://huggingface.co/papers/2606.01348 Authors:

,

,

,

,

,

,

,

,

,

,

,

Abstract

ChartArena presents a comprehensive bilingual benchmark for chart parsing that evaluates models across diverse chart types and visual conditions while providing a unified evaluation framework for fair comparison.

Charts are a primary medium for conveying quantitative and relational information, yet systematically evaluatingchart parsingmodels remains difficult. Existing benchmarks focus on narrow chart types and leave diagrammatic structures such as flowcharts and mind maps largely unaddressed, while models produce outputs in incompatible formats, and datasets rarely include the printed or hand-drawn images encountered in practice. To address these issues, we introduce ChartArena, a comprehensive bilingual benchmark covering eight chart families spanning both numeric charts and diagrammatic structures, each evaluated across threevisual scenarios: digital renderings, printed photos, and hand-drawn photos. The dataset is built via ahuman-agent collaborative annotationpipeline withmulti-stage human verificationto ensure annotation reliability. To enable fair cross-model comparison, we further design aformat-agnostic evaluationprotocol that maps heterogeneous outputs into twocanonical semantic spaces, anormalized triple viewand adirected graph view, and scores them withstructure-aware metrics. Through extensive evaluation of 26 leadingMLLMs, we observe three consistent findings: (i) frontier proprietary models such as Gemini 3.1 Pro lead overall, yet the strongest open-source systems are rapidly closing the gap; (ii) document parsing models handle numeric charts reasonably but fall sharply behind on diagrammatic structures; and (iii) expert chart parsers remain limited to narrow chart families. Across all models, radar charts and hand-drawn scenarios stay especially challenging. These findings show that ChartArena exposes clear capability gaps and provides a unified foundation for future progress. ChartArena is publicly available at https://github.com/pspdada/ChartArena.

View arXiv pageView PDFGitHub0Add to collection

Get this paper in your agent:

hf papers read 2606\.01348

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.01348 in a model README.md to link it from this page.

Datasets citing this paper1

#### psp-dada/ChartArena Viewer• Updatedabout 12 hours ago • 2.38k • 20

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.01348 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

ChartWalker: Benchmarking the Cross-Chart RAG Task

Hugging Face Daily Papers

ChartWalker introduces a novel framework for cross-chart retrieval-augmented generation (RAG) using hierarchical knowledge graph construction and structure-aware sampling. It releases a challenging benchmark (ChartWalker-Bench) and an agentic baseline (ChartWalker-Agent), revealing significant performance gaps in current RAG paradigms.

MIT researchers teach AI models to interpret charts

MIT News — Artificial Intelligence

MIT researchers developed ChartNet, a dataset of over a million charts, to train vision-language models to interpret charts more accurately. Their open-source models outperform much larger commercial models on chart understanding tasks.

Chart-Supported or Model-Supplied? Examining MLLM-Generated Claims for Accessible Visualization

arXiv cs.AI

This paper explores how well multimodal large language models (MLLMs) can generate claims about visualizations, distinguishing between chart-supported evidence and model-supplied interpretation. Through experiments with four input conditions and three MLLMs, it finds that accessible chart context shifts models toward more direct claims, but real-world significance sections remain predominantly speculative.