Can AI Draw Science? A Benchmark for Evaluating Scientific Figure Generation by Text-to-Image and Multimodal Models

# Can AI Draw Science? A Benchmark for Evaluating Scientific Figure Generation by Text-to-Image and Multimodal Models
Source: [https://arxiv.org/html/2606.28406](https://arxiv.org/html/2606.28406)
\(June 2026\)

###### Abstract

Text\-to\-image and multimodal generative models are increasingly used to produce scientific figures—mechanism diagrams, experimental\-design schematics, conceptual frameworks, and graphical abstracts\. Yet existing image\-generation benchmarks \(e\.g\., GenEval, T2I\-CompBench, DPG\-Bench\) evaluate*natural*images and measure compositionality, object counting, or photorealism\. None of them measure the properties that determine whether a generated*scientific*figure is usable: correct and legible text labels, faithful depiction of entities and their relations, coherent diagrammatic structure, and adherence to disciplinary drawing conventions\. We introduceSciDraw\-Bench, a benchmark of 32 structured scientific\-figure generation tasks spanning eight figure types and ten disciplines, where each task pairs a natural\-language prompt with a machine\-checkable specification of required labels, relations, components, conventions, and negative constraints\. We propose a four\-dimensional evaluation protocol—Text Fidelity \(OCR\-based label recall and character error rate\), Semantic Correctness \(vision–language\-model judging against the specification\), Structural Quality, and Convention Adherence—together with a meta\-evaluation protocol and a preliminary inter\-judge reliability analysis \(human\-rating validation is ongoing\)\. We evaluate a domain\-specific system,SciDraw AI, against representative general\-purpose text\-to\-image models, and outline a code\-to\-figure baseline \(LLM\-generated vector graphics\) as a planned extension\. The task specifications and evaluation harness are available from the author on request, supporting reproducible measurement of progress on scientific figure generation\. In a pilot over all eight figure types, the domain\-specific system substantially outperforms the general\-purpose text\-to\-image baselines on every dimension and figure type, with the largest gaps on semantic correctness and disciplinary\-convention adherence; text fidelity remains the hardest dimension for all systems\.

Keywords:text\-to\-image generation, scientific figures, benchmark, evaluation, vision–language models, reproducibility

## 1Introduction

Scientific figures are a primary medium of scholarly communication: mechanism diagrams, experimental\-design schematics, conceptual frameworks, and graphical abstracts compress complex ideas into a form that supports peer review, replication, and teaching\(Tufte,[2001](https://arxiv.org/html/2606.28406#bib.bib16); Rougier et al\.,[2014](https://arxiv.org/html/2606.28406#bib.bib12)\)\. Producing them well has traditionally required substantial time and specialized tooling\. The rapid improvement of text\-to\-image \(T2I\) diffusion models\(Rombach et al\.,[2022](https://arxiv.org/html/2606.28406#bib.bib11); Saharia et al\.,[2022](https://arxiv.org/html/2606.28406#bib.bib13)\), autoregressive image models\(Yu et al\.,[2022](https://arxiv.org/html/2606.28406#bib.bib18)\), and multimodal foundation models\(Betker et al\.,[2023](https://arxiv.org/html/2606.28406#bib.bib1); Team et al\.,[2023](https://arxiv.org/html/2606.28406#bib.bib15)\)has made it plausible that such figures could be generated directly from natural\-language descriptions, and a number of systems now target this use case\.

Whether current models can actually draw*science*, however, is an open empirical question—and one that existing benchmarks do not answer\. The dominant T2I evaluations measure properties of*natural*images: GenEval\(Ghosh et al\.,[2023](https://arxiv.org/html/2606.28406#bib.bib5)\)and T2I\-CompBench\(Huang et al\.,[2023](https://arxiv.org/html/2606.28406#bib.bib9)\)probe object presence, counting, color binding, and spatial composition; DPG\-Bench\(Hu et al\.,[2024](https://arxiv.org/html/2606.28406#bib.bib8)\)measures dense prompt following; HEIM\(Lee et al\.,[2023](https://arxiv.org/html/2606.28406#bib.bib10)\)aggregates aesthetics, alignment, robustness, and bias; and FID\(Heusel et al\.,[2017](https://arxiv.org/html/2606.28406#bib.bib7)\)and CLIPScore\(Hessel et al\.,[2021](https://arxiv.org/html/2606.28406#bib.bib6)\)score distributional realism and image–text alignment\. None of these capture what makes a scientific figure*usable*:

1. \(i\)Text fidelity\.Scientific figures are label\-dense\. A mechanism diagram with garbled or misspelled gene names is worthless, yet T2I models are notoriously unreliable at in\-image text\(Chen et al\.,[2023](https://arxiv.org/html/2606.28406#bib.bib2); Yang et al\.,[2023](https://arxiv.org/html/2606.28406#bib.bib17)\)\.
2. \(ii\)Semantic correctness\.The right entities must appear in the right relations\. “PTEN inhibits PIP3” must be drawn as inhibition, not activation\.
3. \(iii\)Structural quality\.Reading order, arrow directionality, and grouping must communicate the intended logic\.
4. \(iv\)Convention adherence\.Disciplines have visual grammars \(membranes as bilayers, inhibition as blunt arrows, counter\-current condenser flow\) that domain readers expect\.

We address this gap withSciDraw\-Bench\. Our contributions are:

1. \(1\)A benchmark datasetof 32 scientific\-figure generation tasks across eight figure types and ten disciplines\. Crucially, the dataset consists of*tasks*, not reference images: each task is a prompt paired with a machine\-checkable specification \(required labels, relations, components, conventions, negative constraints\)\. There is no single ground\-truth image, because a specification admits many correct renderings \(Section[3](https://arxiv.org/html/2606.28406#S3)\)\.
2. \(2\)An evaluation protocoltailored to scientific figures, combining OCR\-based Text Fidelity, vision–language\-model \(VLM\) judging of Semantic Correctness, and rubric\-based Structural Quality and Convention Adherence \(Section[4](https://arxiv.org/html/2606.28406#S4)\)\.
3. \(3\)A meta\-evaluation protocolfor validating the automatic metrics against human expert ratings using rank correlation and inter\-annotator agreement—the step that distinguishes a measurement instrument from an ad\-hoc tool comparison—together with a preliminary inter\-judge reliability analysis \(Section[4](https://arxiv.org/html/2606.28406#S4)\); the human\-rating study is ongoing\.
4. \(4\)A pilot comparative studyof a domain\-specific system against representative general\-purpose text\-to\-image baselines, with a qualitative error taxonomy \(Sections[5](https://arxiv.org/html/2606.28406#S5)–[8](https://arxiv.org/html/2606.28406#S8)\)\.

The task specifications and evaluation harness are available from the author on request so that the benchmark can be re\-run as models evolve\. This manuscript fixes the benchmark and protocol and reports a pilot evaluation across all eight figure types \(Sections[6](https://arxiv.org/html/2606.28406#S6)–[8](https://arxiv.org/html/2606.28406#S8)\); the human meta\-evaluation is ongoing\.

## 2Related Work

#### Text\-to\-image evaluation\.

Early evaluation of generative image models relied on distributional metrics such as FID\(Heusel et al\.,[2017](https://arxiv.org/html/2606.28406#bib.bib7)\)and on image–text alignment scores such as CLIPScore\(Hessel et al\.,[2021](https://arxiv.org/html/2606.28406#bib.bib6)\)\. As prompt following became the bottleneck, prompt\-centric benchmarks emerged: PartiPrompts\(Yu et al\.,[2022](https://arxiv.org/html/2606.28406#bib.bib18)\), GenEval\(Ghosh et al\.,[2023](https://arxiv.org/html/2606.28406#bib.bib5)\), T2I\-CompBench\(Huang et al\.,[2023](https://arxiv.org/html/2606.28406#bib.bib9)\), and DPG\-Bench\(Hu et al\.,[2024](https://arxiv.org/html/2606.28406#bib.bib8)\)measure object presence, counting, attribute binding, spatial relations, and dense prompt adherence\. HEIM\(Lee et al\.,[2023](https://arxiv.org/html/2606.28406#bib.bib10)\)broadens evaluation to multiple human\-aligned dimensions\. All of these target natural\-image generation; none define a notion of*diagrammatic correctness*or measure in\-image scientific text, which is the focus of SciDraw\-Bench\.

#### Text rendering in images\.

The difficulty of rendering legible text has motivated specialized methods such as TextDiffuser\(Chen et al\.,[2023](https://arxiv.org/html/2606.28406#bib.bib2)\)and GlyphControl\(Yang et al\.,[2023](https://arxiv.org/html/2606.28406#bib.bib17)\), and corresponding evaluations based on OCR accuracy\. We adopt an OCR\-based label\-recall metric in the same spirit, but apply it to the domain\-specific labels demanded by a figure specification rather than to free\-form text rendering\.

#### Model\- and VLM\-as\-judge\.

Using strong models to judge generated outputs is now standard for open\-ended tasks\(Zheng et al\.,[2023](https://arxiv.org/html/2606.28406#bib.bib19)\)\. We use a VLM\-as\-judge ensemble for the semantic and structural dimensions, force structured per\-item verdicts against an explicit rubric, and report inter\-judge agreement as a reliability check; validation against human ratings is part of our protocol and is ongoing\.

#### Scientific visualization and tools\.

Principles of effective scientific figures are long established\(Tufte,[2001](https://arxiv.org/html/2606.28406#bib.bib16); Rougier et al\.,[2014](https://arxiv.org/html/2606.28406#bib.bib12)\), and tools such as BioRender and domain\-specific AI platforms\(SciDraw AI,[2025](https://arxiv.org/html/2606.28406#bib.bib14)\)aim to lower the cost of producing them\. Prior policy\-oriented work has surveyed the editorial acceptability of AI\-generated figures\(Chen,[2026](https://arxiv.org/html/2606.28406#bib.bib3)\); here we instead ask the orthogonal, measurable question of how*good*current systems are at the task\.

## 3SciDraw\-Bench

### 3\.1Design principle: tasks, not images

A scientific figure specification is satisfied by many distinct images\. A correct CAR\-T mechanism diagram may place the T cell on the left or the right, use different iconography for the receptor, and still be fully correct\. Pinning a single reference image as ground truth would therefore penalize valid variation and reward stylistic mimicry\. SciDraw\-Bench instead defines each item as a*generation task*: a natural\-language prompt \(the input given verbatim to every system\) plus a structured*specification*of what any correct output must contain\. Evaluation scores a generated image against the specification, not against a reference image\.

### 3\.2Task schema

Each task is a JSON record with the following fields:

\{

"id":"mech\-001",

"figure\_type":"mechanism",

"discipline":"immunology",

"difficulty":"easy",

"prompt":"DrawamolecularmechanismdiagramofaCAR\-Tcell\.\.\.",

"required\_labels":\["CAR\-Tcell","HER2","scFv","CD3z",\.\.\.\],

"required\_relations":\[\["scFv","binds","HER2"\],\.\.\.\],

"required\_components":\["T\-cellwithmembranebilayer",\.\.\.\],

"discipline\_conventions":\["receptorasY\-shapedtransmembrane",\.\.\.\],

"negative\_constraints":\["nophotorealisticmicrographs",\.\.\.\]

\}

Each field maps to an evaluation dimension \(Section[4](https://arxiv.org/html/2606.28406#S4)\):required\_labels→\\rightarrowText Fidelity;required\_relationsandrequired\_components→\\rightarrowSemantic Correctness; layout/ordering cues in the prompt→\\rightarrowStructural Quality;discipline\_conventionsandnegative\_constraints→\\rightarrowConvention Adherence\.

### 3\.3Coverage and statistics

SciDraw\-Bench contains 32 tasks, balanced as four tasks for each of eight figure types \(Table[1](https://arxiv.org/html/2606.28406#S3.T1)\)\. Each figure type contains one*easy*, two*medium*, and one*hard*task, where difficulty scales with the number of required labels, the density of required relations, and the strictness of conventions \(8 easy / 16 medium / 8 hard overall\)\. Tasks span ten disciplines—biomedicine, cell biology, immunology, chemistry, physics, computer science, environmental science, materials science, neuroscience, and social science—so that no single visual idiom dominates\. The set is intentionally compact and fully hand\-authored to keep specifications accurate and auditable; the schema and harness are designed so the suite can be extended without changing the protocol\.

Table 1:SciDraw\-Bench figure types \(4 tasks each; 32 total\)\. Each type spans easy/medium/hard difficulty\.

## 4Evaluation Protocol

We score each generated image on four dimensions, each normalized to\[0,1\]\[0,1\], plus a composite\. For each \(system, task\) pair we generatek=3k\{=\}3samples to account for stochasticity and report both best\-of\-kkand mean\. All model versions and generation timestamps are logged for reproducibility\.

### 4\.1Text Fidelity \(TF\)

We extract the text tokens present in the image by OCR; the harness supports PaddleOCR\(Du et al\.,[2020](https://arxiv.org/html/2606.28406#bib.bib4)\)and Tesseract backends, and the pilot reported here uses a vision–language\-model transcription pass\. For each string inrequired\_labels, we test for a fuzzy match against the OCR tokens using a normalized Levenshtein distance \(we accept a match when the normalized distance is below a thresholdτ\\tau; the harness usesτ=0\.30\\tau\{=\}0\.30\)\. We report*label recall*\(fraction of required labels found\) and the mean character error rate over matched labels, and combine them into

TF=α⋅recall\+\(1−α\)​\(1−CER\),α=0\.7\.\\mathrm\{TF\}=\\alpha\\cdot\\text\{recall\}\+\(1\-\\alpha\)\\,\(1\-\\text\{CER\}\),\\qquad\\alpha=0\.7\.\(1\)

### 4\.2Semantic Correctness \(SC\)

A VLM\-as\-judge is shown the generated image and the task specification and returns a per\-item yes/no verdict for each entry ofrequired\_relationsandrequired\_components: is the relation depicted with the correct directionality, is the component present, and are there hallucinated entities that contradict the specification? SC is the fraction of specification items satisfied\. To reduce single\-judge bias we use an ensemble of at least two judge models and average their item\-level verdicts\. The exact rubric prompt is fixed and included in the harness so that judgments are auditable and reportable\.

### 4\.3Structural Quality \(SQ\) and Convention Adherence \(CA\)

SQ is a VLM\-judged 1–5 rubric covering reading order, arrow directionality, grouping coherence, and absence of overlap clutter\. CA is a VLM\-judged 1–5 rubric checking the task’sdiscipline\_conventionsand penalizing violations ofnegative\_constraints\. Both are linearly mapped to\[0,1\]\[0,1\]\.

### 4\.4Composite

The composite score is a weighted mean

Overall=wTF​TF\+wSC​SC\+wSQ​SQ\+wCA​CA,\\mathrm\{Overall\}=w\_\{\\mathrm\{TF\}\}\\mathrm\{TF\}\+w\_\{\\mathrm\{SC\}\}\\mathrm\{SC\}\+w\_\{\\mathrm\{SQ\}\}\\mathrm\{SQ\}\+w\_\{\\mathrm\{CA\}\}\\mathrm\{CA\},\(2\)with default weights\(0\.30,0\.30,0\.20,0\.20\)\(0\.30,0\.30,0\.20,0\.20\)\. We report a sensitivity analysis over weights in Appendix[A](https://arxiv.org/html/2606.28406#A1)\.

### 4\.5Meta\-evaluation

Automatic metrics are only useful if they track human judgment\. The protocol therefore collects human ratings from at least three annotators on a stratified subset of generated images, on the same four dimensions \(1–5\), and reports \(i\) Spearmanρ\\rhoand Kendallτ\\taubetween automatic and mean\-human scores per dimension, and \(ii\) inter\-annotator agreement via Krippendorff’sα\\alphaand ICC\(2,kk\)\. Strong rank correlation and acceptable agreement are required before the automatic scores are interpreted as calibrated measurements rather than estimates\. This human study is ongoing; as an interim reliability check we report inter\-judge agreement between the two automatic judges \(Section[7](https://arxiv.org/html/2606.28406#S7)\)\.

## 5Experimental Setup

#### Systems under test\.

We compare a domain\-specific scientific\-illustration platform,SciDraw AI\([https://sci\-draw\.com](https://sci-draw.com/)\)\(SciDraw AI,[2025](https://arxiv.org/html/2606.28406#bib.bib14)\), against representative general\-purpose text\-to\-image models \(Table[2](https://arxiv.org/html/2606.28406#S5.T2)\)\. Because SciDraw AI is a model\-agnostic platform—its visual backend is configurable—we compare it against standalone text\-to\-image models rather than treating any single foundation model as a fixed rival\. The question is whether the platform’s domain tooling helps on the dimensions general models struggle with\. A code\-to\-figure baseline \(an LLM emitting SVG/TikZ/Mermaid, subsequently rendered\) and additional diffusion models are planned extensions of the suite\.

Table 2:Systems evaluated in the pilot\. Specific model versions and generation dates are logged in the run configuration\.
#### Protocol controls\.

Every system receives the identical task prompt; we apply no benchmark\-side prompt engineering\. The domain\-specific platform additionally applies its own production scientific\-illustration instruction layer, which is a property of the system under test rather than a benchmark intervention\. The pilot usesk=2k\{=\}2samples per task \(the full protocol specifiesk=3k\{=\}3\)\. The two judge models \(GPT\-4o and Gemini\) are drawn from families disjoint from the text\-to\-image baselines; we additionally report inter\-judge agreement \(Section[7](https://arxiv.org/html/2606.28406#S7)\) as a reliability check\.

## 6Results

#### Qualitative pilot examples\.

Figure[1](https://arxiv.org/html/2606.28406#S6.F1)shows SciDraw\-Bench tasks from four different figure types rendered by the domain\-specific system, illustrating the range of scientific illustration the benchmark targets—concept maps, apparatus schematics, technical roadmaps, and graphical abstracts—and the level of text and convention fidelity attainable on these tasks\.

![Refer to caption](https://arxiv.org/html/2606.28406v1/figures/show-carboncycle.png)\(a\)Concept map \(cmap\-001, carbon cycle\)
![Refer to caption](https://arxiv.org/html/2606.28406v1/figures/show-interferometer.png)\(b\)Apparatus schematic \(appa\-002, Michelson interferometer\)
![Refer to caption](https://arxiv.org/html/2606.28406v1/figures/show-quantum.png)\(c\)Technical roadmap \(road\-003, fault\-tolerant QC\)
![Refer to caption](https://arxiv.org/html/2606.28406v1/figures/show-microbiome.png)\(d\)Graphical abstract \(gabs\-002, gut–brain axis\)

Figure 1:Representative SciDraw\-Bench tasks across four figure types, rendered by the domain\-specific system \(SciDraw AI\)\. Labels, relations, and disciplinary conventions are largely satisfied\. These are single \(k=1k\{=\}1\) unedited outputs\.Table 3:Main results: mean dimension scores and composite score per system, averaged overk=2k\{=\}2samples on all 8 pilot tasks \(16 images/system\), scored by the two\-judge ensemble\. Best per column in bold\.Table[3](https://arxiv.org/html/2606.28406#S6.T3)reports mean scores over the 16 images per system; Table[5](https://arxiv.org/html/2606.28406#A2.T5)\(Appendix\) breaks the composite down by figure type\. Three findings stand out\.

The domain\-specific platform dominates the general\-purpose baselines\.SciDraw AI scores 0\.91 on the composite, far above Qwen\-Image \(0\.64\) and Ideogram v3 \(0\.57\)\. The separation is largest on the judged dimensions: the general\-purpose models frequently omit or misorder required entities \(SC0\.800\.80/0\.640\.64vs0\.980\.98\) and violate disciplinary conventions \(CA0\.510\.51/0\.480\.48vs0\.980\.98\), and at times return decorative, off\-task output with no diagrammatic structure, whereas SciDraw AI satisfies nearly all specification items\.

Text Fidelity is the universal bottleneck\.No system exceeds 0\.74 on TF—markedly lower than the SC/SQ/CA scores—confirming that rendering correct, legible in\-image scientific text remains the hardest part of the task even for the strongest system \(cf\. the garbled and misspelled labels in Figure[2](https://arxiv.org/html/2606.28406#S8.F2)\)\.

The advantage holds across every figure type\.SciDraw AI achieves the highest composite on all eight figure types \(Table[5](https://arxiv.org/html/2606.28406#A2.T5)\), with its largest margins on communication\-oriented figures—concept maps \(1\.001\.00\), conceptual frameworks \(0\.950\.95\), graphical abstracts \(0\.970\.97\), and technical roadmaps \(0\.890\.89\)\. This matches our pre\-registered expectation \(H3\) that domain\-specific tooling helps most on template\-like, communicative figures, and that general T2I models score lowest on Text Fidelity \(H1\)\.

## 7Meta\-Evaluation

A full meta\-evaluation validates the automatic metrics against human expert ratings; collecting those ratings is ongoing and we report it as future work\. As a reliability check available now, Table[4](https://arxiv.org/html/2606.28406#S7.T4)reports agreement between the two independent judges \(OpenAI GPT\-4o and Gemini\) on the three judged dimensions over then=46n\{=\}46scored image pairs\. Agreement is moderate \(Pearsonrrbetween 0\.58 and 0\.63; mean absolute difference≤0\.21\\leq 0\.21on a\[0,1\]\[0,1\]scale\), which supports averaging them as an ensemble while motivating the human validation below\. Human\-validated correlations remain to be established before the automatic scores are interpreted as calibrated measurements rather than internally consistent estimates\.

Table 4:Inter\-judge agreement between the two independent judges on the judged dimensions \(n=46n\{=\}46image pairs\)\. Text Fidelity is computed programmatically from OCR and is not judge\-scored\. Human meta\-evaluation is future work\.
## 8Error Analysis

The general\-purpose baselines fail in characteristic ways, illustrated in Figure[2](https://arxiv.org/html/2606.28406#S8.F2)on the taskmech\-002\(the PI3K/AKT/mTOR cascade\)\. One model produces a plausibly laid\-out diagram whose text is badly garbled \(e\.g\., “PI3P/AKT/MOTOR”, “mTJRC1”, “mTERC1”\) with misspelled and duplicated labels; the other returns an off\-task decorative abstraction with no diagrammatic content or labels at all\. By contrast, the domain\-specific system renders the same pathway with all required labels \(RTK, PI3K, PIP2/PIP3, PTEN, AKT, mTORC1, S6K1, 4E\-BP1\) legible and correctly spelled, appropriate activation/inhibition glyphs, and a membrane bilayer \(the comparable mechanism output appears among the examples in Figure[1](https://arxiv.org/html/2606.28406#S6.F1)for other figure types\)\.

![Refer to caption](https://arxiv.org/html/2606.28406v1/figures/cmp-qwen.png)\(a\)Garbled text \(Qwen\-Image\)
![Refer to caption](https://arxiv.org/html/2606.28406v1/figures/cmp-ideogram.png)\(b\)Off\-task rendering \(Ideogram v3\)

Figure 2:Characteristic failure modes of general\-purpose text\-to\-image models on the identical taskmech\-002\(PI3K/AKT/mTOR signaling\), single \(k=1k\{=\}1\) unedited outputs: \(a\) a diagram\-like layout with severely garbled, misspelled labels, and \(b\) an off\-task decorative abstraction lacking any diagrammatic structure or text\.These observations motivate the failure taxonomy the full evaluation quantifies: \(i\)*garbled text*—misspelled or unreadable labels \(Text Fidelity\); \(ii\)*off\-task rendering*—decorative output lacking diagrammatic structure \(Structural Quality\); \(iii\)*hallucinated entities*—elements absent from the specification \(Semantic Correctness\); \(iv\)*relation inversion*—inhibition drawn as activation or reversed arrows \(Semantic Correctness\); and \(v\)*convention violation*—e\.g\., membranes not drawn as bilayers \(Convention Adherence\)\. The automatic metrics in Section[4](https://arxiv.org/html/2606.28406#S4)are constructed to score exactly these properties, and each flagged example is linked to the specific specification item it violates, making the failure mode reproducible rather than anecdotal\. We caution that these are single\-task illustrations; per\-system rankings \(Section[6](https://arxiv.org/html/2606.28406#S6)\) are established by the aggregate scores over all 16 images per system, and the human\-validated meta\-evaluation remains ongoing\.

## 9Discussion

SciDraw\-Bench reframes “can AI draw science?” as a measurable question with per\-dimension answers rather than a holistic impression\. By scoring against specifications rather than reference images, it credits valid visual variation while still enforcing correctness\. The four dimensions also map onto distinct intervention points: Text Fidelity is bounded by in\-image text rendering; Semantic Correctness by prompt grounding; and Convention Adherence by domain knowledge—suggesting that hybrid or domain\-specialized pipelines may dominate on different axes\. The results connect directly to editorial guidance on AI\-generated figures\(Chen,[2026](https://arxiv.org/html/2606.28406#bib.bib3)\): the dimension on which every system remains weakest—Text Fidelity—is precisely the one where human review and disclosure remain non\-negotiable, since even the strongest models still misspell or garble in\-image labels\.

## 10Limitations

Several limitations apply\. \(i\)*Pilot scale*: the suite contains 32 tasks; the evaluation reported here is a pilot over 8 tasks \(one per figure type\) and 3 systems atk=2k\{=\}2, giving broad type coverage but limited statistical power within cells\. The protocol is designed to scale to the full suite and more systems without change\. \(ii\)*Non\-exhaustive comparison*: we compare against widely\-available general\-purpose text\-to\-image baselines; this is not an exhaustive comparison, and strong commercial multimodal image models can match domain\-specific output quality on several figure types\. A controlled comparison against such models, and against the planned code\-to\-figure baseline, is left to future work\. \(iii\)*Stochasticity*: we mitigate but do not eliminate sampling variance viak=2k\{=\}2and mean reporting\. \(iv\)*Judge bias and saturation*: VLM judges may exhibit systematic leniency; the near\-perfect judged scores of the leading system suggest the 1–5 rubrics saturate at the high end and may under\-resolve differences between strong systems\. We use a two\-judge ensemble and report inter\-judge agreement, but human validation remains future work\. \(v\)*Specification subjectivity*: “correct” conventions encode authorial judgment; specifications are available for scrutiny and revision\. \(vi\)*Temporal validity*: model versions change quickly, so all results are timestamped and the harness is available for re\-running\.

## 11Conclusion

We introduced SciDraw\-Bench, the first benchmark, to our knowledge, dedicated to scientific figure generation, together with a four\-dimensional evaluation protocol and a meta\-evaluation protocol for validating the automatic metrics against human judgment \(with a preliminary inter\-judge reliability analysis\)\. The benchmark measures the properties that decide whether a generated figure is usable in research—text fidelity, semantic correctness, structure, and convention—that natural\-image benchmarks omit\. The task specifications and evaluation harness are available from the author on request, and we invite the community to extend the suite and re\-run it as models improve\.

## Data and code availability

The SciDraw\-Bench task specifications, the evaluation harness, the exact judge rubric prompts, and the generated artifacts and run configurations are available from the author on reasonable request\.

## References

- Betker et al\. \[2023\]Betker, J\., Goh, G\., Jing, L\., et al\. \(2023\)\.Improving image generation with better captions \(DALL⋅\\cdotE 3\)\.OpenAI technical report\.
- Chen et al\. \[2023\]Chen, J\., Huang, Y\., Lv, T\., Cui, L\., Chen, Q\., and Wei, F\. \(2023\)\.TextDiffuser: Diffusion models as text painters\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Chen \[2026\]Chen, D\. \(2026\)\.AI\-generated figures in academic publishing: Policies, tools, and practical guidelines\.Preprint\.
- Du et al\. \[2020\]Du, Y\., Li, C\., Guo, R\., et al\. \(2020\)\.PP\-OCR: A practical ultra lightweight OCR system\.*arXiv preprint arXiv:2009\.09941*\.
- Ghosh et al\. \[2023\]Ghosh, D\., Hajishirzi, H\., and Schmidt, L\. \(2023\)\.GenEval: An object\-focused framework for evaluating text\-to\-image alignment\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Hessel et al\. \[2021\]Hessel, J\., Holtzman, A\., Forbes, M\., Le Bras, R\., and Choi, Y\. \(2021\)\.CLIPScore: A reference\-free evaluation metric for image captioning\.In*Proceedings of EMNLP*\.
- Heusel et al\. \[2017\]Heusel, M\., Ramsauer, H\., Unterthiner, T\., Nessler, B\., and Hochreiter, S\. \(2017\)\.GANs trained by a two time\-scale update rule converge to a local Nash equilibrium\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Hu et al\. \[2024\]Hu, X\., Wang, R\., Fang, Y\., et al\. \(2024\)\.ELLA: Equip diffusion models with LLM for enhanced semantic alignment \(DPG\-Bench\)\.*arXiv preprint arXiv:2403\.05135*\.
- Huang et al\. \[2023\]Huang, K\., Sun, K\., Xie, E\., Li, Z\., and Liu, X\. \(2023\)\.T2I\-CompBench: A comprehensive benchmark for open\-world compositional text\-to\-image generation\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Lee et al\. \[2023\]Lee, T\., Yasunaga, M\., Meng, C\., et al\. \(2023\)\.Holistic evaluation of text\-to\-image models \(HEIM\)\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Rombach et al\. \[2022\]Rombach, R\., Blattmann, A\., Lorenz, D\., Esser, P\., and Ommer, B\. \(2022\)\.High\-resolution image synthesis with latent diffusion models\.In*Proceedings of the IEEE/CVF CVPR*, pages 10684–10695\.
- Rougier et al\. \[2014\]Rougier, N\. P\., Droettboom, M\., and Bourne, P\. E\. \(2014\)\.Ten simple rules for better figures\.*PLoS Computational Biology*, 10\(9\):e1003833\.
- Saharia et al\. \[2022\]Saharia, C\., Chan, W\., Saxena, S\., et al\. \(2022\)\.Photorealistic text\-to\-image diffusion models with deep language understanding \(Imagen\)\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- SciDraw AI \[2025\]SciDraw AI \(2025\)\.SciDraw AI: An AI\-powered scientific illustration platform\.[https://sci\-draw\.com](https://sci-draw.com/)\.
- Team et al\. \[2023\]Team, G\., Anil, R\., Borgeaud, S\., Wu, Y\., et al\. \(2023\)\.Gemini: A family of highly capable multimodal models\.*arXiv preprint arXiv:2312\.11805*\.
- Tufte \[2001\]Tufte, E\. R\. \(2001\)\.*The Visual Display of Quantitative Information*\.Graphics Press, 2nd edition\.
- Yang et al\. \[2023\]Yang, Y\., Gui, D\., Yuan, Y\., et al\. \(2023\)\.GlyphControl: Glyph conditional control for visual text generation\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Yu et al\. \[2022\]Yu, J\., Xu, Y\., Koh, J\. Y\., et al\. \(2022\)\.Scaling autoregressive models for content\-rich text\-to\-image generation \(Parti\)\.*Transactions on Machine Learning Research*\.
- Zheng et al\. \[2023\]Zheng, L\., Chiang, W\.\-L\., Sheng, Y\., et al\. \(2023\)\.Judging LLM\-as\-a\-judge with MT\-Bench and Chatbot Arena\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.

## Appendix AWeight sensitivity

We recompute the composite under alternative weightings to confirm that conclusions are not artifacts of the default weights\(0\.30,0\.30,0\.20,0\.20\)\(0\.30,0\.30,0\.20,0\.20\)\. SciDraw AI scores0\.920\.92under balanced weights\(0\.25,0\.25,0\.25,0\.25\)\(0\.25,0\.25,0\.25,0\.25\)and0\.860\.86under text\-heavy weights\(0\.5,0\.3,0\.1,0\.1\)\(0\.5,0\.3,0\.1,0\.1\), versus0\.630\.63/0\.660\.66for Qwen\-Image and0\.570\.57/0\.590\.59for Ideogram v3\. The ranking \(SciDraw AI≫\\ggQwen\-Image\>\>Ideogram v3\) is preserved under all three weightings\.

## Appendix BPer\-type breakdown

Table 5:Composite score by figure type \(mean overk=2k\{=\}2samples, ensemble judges\)\. Best per row in bold\. SciDraw AI leads on every figure type, by the widest margins on communication\-oriented figures\.