ChemVA: Advancing Large Language Models on Chemical Reaction Diagrams Understanding
Summary
ChemVA is a framework that addresses the visual and semantic bottlenecks in LLMs for understanding chemical reaction diagrams, achieving 92% structural recognition accuracy and consistent ~20 percentage point gains across nine LLMs.
arXiv:2605.17214v1 Announce Type: new
Abstract: While Large Language Models (LLMs) have revolutionized scientific text processing, they exhibit a significant capability gap when interpreting chemical reaction diagrams. We identify two fundamental bottlenecks restricting current systems: a Visual Deficit, where generic vision encoders struggle to resolve the strict topological connectivity of dense molecular graphs, and a Semantic Disconnect, where standard linear strings, such as SMILES, fail to effectively activate the model's latent chemical reasoning. To bridge these gaps, we propose the Chemical Visual Activation (ChemVA) framework, which employs a Visual Anchor mechanism to ground functional groups via hybrid-granularity detection, followed by a semantic alignment approach that translates visual features into entity names to maximize knowledge activation in LLMs. We evaluate our approach on OCRD-Bench, a newly constructed dataset featuring dense visual-semantic contexts and comprehensive reaction coverage to evaluate the full spectrum from recognition to reasoning. Extensive experiments on OCRD-Bench demonstrate that ChemVA achieves 92.0% structural recognition accuracy. By bridging visual and semantic bottlenecks, our framework delivers a consistent performance gain of approximately 20 percentage points across 9 diverse LLMs, enabling open-weight models to rival proprietary SOTA systems in complex chemical reasoning tasks.
View Cached Full Text
Cached at: 05/19/26, 06:40 AM
# ChemVA: Advancing Large Language Models on Chemical Reaction Diagrams Understanding Source: [https://arxiv.org/html/2605.17214](https://arxiv.org/html/2605.17214) Mingyang RaoCollege of Computer Science and Technology, Zhejiang UniversityHangzhouZhejiangChina[myR001@zju\.edu\.cn](https://arxiv.org/html/2605.17214v1/mailto:[email protected])Kehua FengCollege of Computer Science and Technology, Zhejiang UniversityHangzhouZhejiangChina[kehuafeng@zju\.edu\.cn](https://arxiv.org/html/2605.17214v1/mailto:[email protected]),Zhihui ZhuZJU\-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang UniversityHangzhouZhejiangChina[Zhihui\.Zhu01@outlook\.com](https://arxiv.org/html/2605.17214v1/mailto:[email protected]),Jiangzhen FuDepartment of Chemistry, Fudan UniversityShanghaiChina[23110220019@m\.fudan\.edu\.cn](https://arxiv.org/html/2605.17214v1/mailto:[email protected]),Hao YuDepartment of Chemistry, Fudan UniversityShanghaiChina[hao\_yu@fudan\.edu\.cn](https://arxiv.org/html/2605.17214v1/mailto:[email protected]),Keyan DingZJU\-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang UniversityHangzhouZhejiangChina[dingkeyan@zju\.edu\.cn](https://arxiv.org/html/2605.17214v1/mailto:[email protected])andHuajun Chen†College of Computer Science and Technology, Zhejiang UniversityHangzhouZhejiangChina[huajunsir@zju\.edu\.cn](https://arxiv.org/html/2605.17214v1/mailto:[email protected]) \(2026\) ###### Abstract\. While Large Language Models \(LLMs\) have revolutionized scientific text processing, they exhibit a significant capability gap when interpreting chemical reaction diagrams\. We identify two fundamental bottlenecks restricting current systems: aVisual Deficit, where generic vision encoders struggle to resolve the strict topological connectivity of dense molecular graphs, and aSemantic Disconnect, where standard linear strings \(*e\.g\.*, SMILES\) fail to effectively activate the model’s latent chemical reasoning\. To bridge these gaps, we propose theChemical Visual Activation \(ChemVA\)framework, which employs a novelVisual Anchormechanism to ground functional groups via hybrid\-granularity detection, followed by a semantic alignment approach that translates visual features into entity names to maximize knowledge activation in LLMs\. We evaluate our approach onOCRD\-Bench, a newly constructed dataset featuring dense visual\-semantic contexts and comprehensive reaction coverage to evaluate the full spectrum from recognition to reasoning\. Extensive experiments on OCRD\-Bench demonstrate that ChemVA achieves 92\.0% structural recognition accuracy\. By bridging visual and semantic bottlenecks, our framework delivers a consistent performance gain of approximately 20 percentage points across 9 diverse LLMs, enabling open\-weights models to rival proprietary SOTA systems in complex chemical reasoning tasks\. Large language models, Chemical diagram reasoning, Optical chemical structure recognition ††copyright:none††journalyear:2026††conference:Preprint; 2026;††ccs:Computing methodologies Natural language processing††ccs:Computing methodologies Computer vision††ccs:Applied computing ChemistryFigure 1\.Overview of limitations in LLM\-based chemical reaction diagram understanding and the proposed ChemVA framework\. \(a\) Existing pipelines exhibit a Visual Deficit in Reaction Diagram Parsing and a Semantic Disconnect under SMILES\-based representations\. \(b\) ChemVA bridges these gaps via functional\-group–grounded Reaction Diagram Parsing and Semantic Activation for downstream reasoning\.## 1\.Introduction The integration of Large Language Models \(LLMs\) into scientific discovery pipelines has accelerated progress in “AI for Science”, especially in domains that require complex reasoning, such as organic chemistry\(Boikoet al\.,[2023](https://arxiv.org/html/2605.17214#bib.bib1); Zhanget al\.,[2024b](https://arxiv.org/html/2605.17214#bib.bib2)\)\. General\-purpose models such as GPT\-5 and Gemini2\.5\-pro have shown strong performance on text\-centric chemical tasks, including synthesis planning and property prediction\([A\. M\. Branet al\.\(2025\)](https://arxiv.org/html/2605.17214#bib.bib50);[Y\. Gu and Z\. Liang \(2025\)](https://arxiv.org/html/2605.17214#bib.bib51);[10](https://arxiv.org/html/2605.17214#bib.bib52)\)\. However, the primary modality of chemical knowledge in the literature remains visual,*i\.e\.*, molecular structures and reaction diagrams, and current state\-of\-the\-art models still exhibit a substantial gap on this modality\(Liet al\.,[2025](https://arxiv.org/html/2605.17214#bib.bib15); Guoet al\.,[2023](https://arxiv.org/html/2605.17214#bib.bib3)\)\. This gap can be traced to two fundamental bottlenecks: aVisual Deficitin structural perception and aSemantic Disconnectin knowledge activation \(Figure[1](https://arxiv.org/html/2605.17214#S0.F1)\)\. First, despite rapid progress in vision\-language models, a criticalVisual Deficitremains in the processing of chemical images\. Chemical structures are dense graph objects whose meaning is determined by topological connectivity and stereochemical geometry, rather than by the global semantic cues that CLIP\-style encoders typically emphasize\(Morinet al\.,[2023](https://arxiv.org/html/2605.17214#bib.bib4); Stakeret al\.,[2019](https://arxiv.org/html/2605.17214#bib.bib53)\)\. Prior evaluations\(Qianet al\.,[2023b](https://arxiv.org/html/2605.17214#bib.bib5); Clevertet al\.,[2021](https://arxiv.org/html/2605.17214#bib.bib6)\)show that general\-purpose models often hallucinate atoms, confuse bond types, and degrade under rotations or visually crowded graphs \(*i\.e\.*, molecules with dense functional groups or complex ring systems\)\. In contrast to natural images, where mild distortions may be tolerable, even a single local mistake in a chemical diagram \(*e\.g\.*, interpreting a wedge bond as a dashed bond\) can change molecular identity and invalidate downstream analysis\. Existing Optical Chemical Structure Recognition \(OCSR\) tools, including MolScribe\(Qianet al\.,[2023b](https://arxiv.org/html/2605.17214#bib.bib5)\), Decimer\(Schüttet al\.,[2020](https://arxiv.org/html/2605.17214#bib.bib11); Rajanet al\.,[2023](https://arxiv.org/html/2605.17214#bib.bib24)\), and related methods\(Chenet al\.,[2025](https://arxiv.org/html/2605.17214#bib.bib56); Oldenhofet al\.,[2021](https://arxiv.org/html/2605.17214#bib.bib55); Shahet al\.,[2025](https://arxiv.org/html/2605.17214#bib.bib64); Wanget al\.,[2025](https://arxiv.org/html/2605.17214#bib.bib65); Zhuanget al\.,[2025](https://arxiv.org/html/2605.17214#bib.bib66)\), have improved image\-to\-graph translation, but the dominant formulation remainsatom\-levelprediction that represents molecules as sequences of nodes and edges\. This formulation does not capturesubstructure\-levelperception, where functional groups function as stable semantic units that guide expert interpretation\. Second, even when structure recognition yields a correct linear representation such as SMILES, downstream reasoning in LLMs can still be hindered by what we term aSemantic Disconnect\. This term refers to the mismatch between chemically valid symbolic encodings and the semantic cues that LLMs rely on during inference\. While SMILES provides a precise specification of molecular graphs, it is largely a non\-semantic character sequence for a language model and does not reliably evoke the functional concepts that support chemical reasoning\. Prior evidence indicates that SMILES alone often fails to effectively “activate” latent chemical knowledge in LLMs\(Edwards and others,[2022](https://arxiv.org/html/2605.17214#bib.bib7)\)\. Conversely, chemical entity names \(*e\.g\.*, IUPAC names\(Helleret al\.,[2015](https://arxiv.org/html/2605.17214#bib.bib80)\)\) provide explicit descriptors that more consistently elicit knowledge about reactivity and synthesis\(Runcie and others,[2025](https://arxiv.org/html/2605.17214#bib.bib8); Diao and others,[2023](https://arxiv.org/html/2605.17214#bib.bib9)\)\. Therefore, pipelines that follow “Image→\\toSMILES→\\toReasoning” can underutilize the reasoning capacity of LLMs, because the intermediate representation is not aligned with the semantic abstractions that guide inference\. To bridge these gaps, we propose Chemical Visual Activation \(ChemVA\), a two\-phase framework that comprisesReaction Diagram ParsingandSemantic Activation\. Motivated by the “chunking” mechanism in human cognition\(Gobet and others,[2001](https://arxiv.org/html/2605.17214#bib.bib10)\), the Reaction Diagram Parsing phase uses functional\-group\-grounded vision\-language model \(FG\-VLM\) to ground functional groups asVisual Anchorsthrough reaction deconstruction and hybrid\-granularity visual anchor modeling, enabling topology\-faithful structural reconstruction\. The Semantic Activation phase then performs semantic alignment by resolving the recognized structures into entity names, which are used to construct semantically grounded prompts that more effectively elicit chemical knowledge in downstream LLMs\. To overcome the limitations of existing datasets, which typically emphasize either structural recognition or text\-only reasoning, we constructOCRD\-Bench, a unified multimodal benchmark derived from graduate\-level chemistry entrance exams\. Experiments on OCRD\-Bench show that ChemVA consistently outperforms strong baselines\. In particular, FG\-VLM reduces error accumulation that is common in specialized OCSR pipelines and mitigates structural hallucinations in general\-purpose multimodal models \(*e\.g\.*, spurious atoms/bonds and incorrect connectivity\)\. In addition,Semantic Activationaddresses the Semantic Disconnect and improves downstream reasoning by providing entity\-name–grounded prompts that better align with the semantic abstractions used by LLMs\. Our contributions can be summarized as follows: - •We propose ChemVA, a two\-phase framework that bridges theVisual DeficitandSemantic DisconnectthroughReaction Diagram ParsingandSemantic Activation, improving diagram understanding and downstream reasoning in chemical MLLMs\. - •We develop FG\-VLM, the first hybrid\-granularity OCSR model that grounds functional groups asVisual Anchorsand reconstructs topology\-faithful molecular graphs via Directional Vector Matching, going beyond purely atom\-level serialization\. - •We construct OCRD\-Bench, a challenging multimodal benchmark derived from graduate\-level entrance exams, with broad reaction coverage and a hierarchical evaluation protocol that spans visual perception, chemical knowledge, and mechanistic reasoning\. - •We show that ChemVA consistently improves performance across a wide range of LLMs, yielding substantial gains on complex reasoning tasks\. ## 2\.Related Work ### 2\.1\.Methods for Chemical Reaction Diagram Understanding Prior research on chemical reaction diagram understanding largely falls into two lines: \(i\) symbolic serialization methods that “digitize” structures from images, and \(ii\) multimodal LLMs that attempt to reason directly from visual inputs\. A long\-standing direction is symbolic serialization via OCSR, which converts raster diagrams into machine\-readable symbolic forms such as molecular graphs or SMILES\. Representative systems \(*e\.g\.*, MolScribe\(Qianet al\.,[2023b](https://arxiv.org/html/2605.17214#bib.bib5)\), RxnScribe\(Qianet al\.,[2024](https://arxiv.org/html/2605.17214#bib.bib12)\), OpenChemIE\(Fanet al\.,[2024](https://arxiv.org/html/2605.17214#bib.bib77)\), and DECIMER\(Schüttet al\.,[2020](https://arxiv.org/html/2605.17214#bib.bib11); Rajanet al\.,[2023](https://arxiv.org/html/2605.17214#bib.bib24)\), ReactionDataExtractor1\.0/2\.0\(Wilary and Cole,[2021](https://arxiv.org/html/2605.17214#bib.bib81),[2023](https://arxiv.org/html/2605.17214#bib.bib82)\)\) typically operate at atom\-level granularity and formulate recognition as structured prediction \(graph prediction or graph\-to\-text encodings\(Yuet al\.,[2024](https://arxiv.org/html/2605.17214#bib.bib59)\)\)\. While these models can achieve strong structural accuracy on clean inputs, they primarily function as semantics\-agnostic digitizers: the output symbols are chemically valid but do not explicitly expose higher\-level semantic units \(*e\.g\.*, functional groups\) that are critical for robust interpretation under rotation/crowding and for downstream reasoning\. Recent work couples vision encoders with LLMs to enable end\-task reasoning\.*Implicit alignment*approaches \(*e\.g\.*, ChemVLM\(Liet al\.,[2024](https://arxiv.org/html/2605.17214#bib.bib42)\)\) align generic vision backbones with chemical text through instruction tuning, but often suffer a visual deficit when faced with dense molecular graphs, leading to structural hallucinations relative to specialized OCSR tools\. Another line,*external retrieval*\(*e\.g\.*, MolMole\(Researchet al\.,[2025](https://arxiv.org/html/2605.17214#bib.bib48)\), Doc2SAR\(Zhuanget al\.,[2025](https://arxiv.org/html/2605.17214#bib.bib66)\)\), grounds LLM outputs by extracting or retrieving molecular facts from documents and databases; this can improve factuality for lookup\-like tasks, but it relies on external resources rather than strengthening intrinsic visual understanding and generative reasoning\. Our framework bridges these paradigms with \(i\) a hybrid\-granularity visual module that grounds*functional groups*as visual anchors for more robust structure understanding beyond pure atom\-level serialization, and \(ii\) a semantic alignment workflow that translates grounded visual evidence into*entity names*to maximize knowledge activation in downstream LLMs, without requiring database retrieval\. ### 2\.2\.Benchmarks for Chemical Reaction Diagram Understanding Existing benchmarks typically isolate specific stages of the chemical reaction diagram understanding pipeline, failing to evaluate end\-to\-end capabilities\. Recognition\-centric benchmarks \(*e\.g\.*, standard USPTO subsets\(Oldenhofet al\.,[2021](https://arxiv.org/html/2605.17214#bib.bib55)\)\) focus exclusively on Image\-to\-SMILES translation accuracy, ignoring downstream chemical validity\. Text\-centric reasoning benchmarks, such as SciBench\(Wanget al\.,[2024](https://arxiv.org/html/2605.17214#bib.bib31)\)and ChemBench\(Zhanget al\.,[2024a](https://arxiv.org/html/2605.17214#bib.bib43)\), assess knowledge but primarily use text inputs, bypassing visual perception challenges\. While recent multimodal benchmarks like MMCR\(Liet al\.,[2024](https://arxiv.org/html/2605.17214#bib.bib42)\)and MaCBench\(Alamparaet al\.,[2024](https://arxiv.org/html/2605.17214#bib.bib73)\)introduce visual reasoning, they are predominantly limited to multiple\-choice questions or document\-level element detection\. Crucially, despite recent progress in molecular and reaction comprehension tasks\([N\. T\. Runcie, C\. M\. Deane, and F\. Imrie \(2025\)](https://arxiv.org/html/2605.17214#bib.bib74);[3](https://arxiv.org/html/2605.17214#bib.bib76);[R\. Xu, Y\. Zhang, Q\. Wang, C\. Edwards, and H\. Ji \(2025\)](https://arxiv.org/html/2605.17214#bib.bib21);[23](https://arxiv.org/html/2605.17214#bib.bib75)\), there remains a significant scarcity of benchmarks that specifically bridge the gap between visual reaction recognition and deep mechanistic reasoning\(Yueet al\.,[2024](https://arxiv.org/html/2605.17214#bib.bib67); Luet al\.,[2022](https://arxiv.org/html/2605.17214#bib.bib69)\)\. Derived from graduate\-level entrance examinations, our OCRD\-Bench is the first benchmark to demand end\-to\-end generative reasoning, requiring models to independently parse complex reaction diagrams and generate multi\-step mechanism explanations\. ## 3\.Method Figure 2\.Overview of the ChemVA framework\.Stage 1: Reaction Diagram Parsing\.\(a\)Reaction Diagram Deconstruction: FG\-VLM localizes diagram elements \(*e\.g\.*, reactants, conditions, arrows\) and crops molecular regions\. \(b\)Visual Anchor Modeling: FG\-VLM predicts a hybrid\-granularity graph \(functional groups and residual atoms\) with bonds, and regresses group attachment anchors as pixel keypoints\. \(c\)Directional Vector Matching: Group templates are oriented and attached to anchors to instantiate a topology\-faithful molecular graph\. \(d\)Reaction Reconstruction: The recovered graphs are converted to SMILES and parsed into a structured reaction sequence\.Stage 2: Semantic Activation\.\(e\)Entity Resolution: SMILES are resolved to IUPAC/common names via PubChem\. \(f\)Activation Prompting: The names are used to construct semantically grounded \(name\-first\) prompts that more reliably elicit chemical knowledge for downstream reasoning\.This section presents the proposedChemVAframework for improving LLM understanding of chemical reaction diagrams\. As shown in Figure[2](https://arxiv.org/html/2605.17214#S3.F2), given a reaction image, ChemVA reconstructs a topology\-faithful molecular graph and its reaction semantics through two main phases: \(1\)Diagram\-to\-Reaction Parsing\(Sec\.[3\.1](https://arxiv.org/html/2605.17214#S3.SS1)\), where we proposeFG\-VLMto progressively recover the structured representation by performing \(i\)reaction deconstruction, \(ii\)hybrid\-granularity visual anchor modelingfor node and anchor prediction, and \(iii\)structural reconstructionvia directional vector matching, followed by \(iv\)reaction reconstructionto assemble the complete reaction sequence; and \(2\)Semantic Activation\(Sec\.[3\.2](https://arxiv.org/html/2605.17214#S3.SS2)\), which translates the recognized structure into semantically grounded prompts to elicit chemical knowledge from LLMs for downstream reasoning\. Finally, we detail thedata construction and trainingprocedure \(Sec\.[3\.3](https://arxiv.org/html/2605.17214#S3.SS3)\) used to equip FG\-VLM with hybrid\-granularity perception via a curriculum\-style strategy\. ### 3\.1\.Reaction Diagram Parsing via FG\-VLM Given a chemical reaction diagram imageII, our goal is to recover a structured reaction representation that supports downstream chemical visual reasoning\. Prior approaches typically cast diagram understanding as atom\-level sequence generation,*i\.e\.*, emitting an ordered list of atomsVatomV\_\{\\text\{atom\}\}and their bonds\(Qianet al\.,[2023b](https://arxiv.org/html/2605.17214#bib.bib5)\)\. However, organic molecules contain recurring rigid motifs whose semantics are determined at the substructure level; purely atom\-level generation therefore produces long outputs and is prone to local errors that propagate to global topology\. To address this, we present functional\-group\-grounded vision\-language model \(FG\-VLM\), which performs end\-to\-end diagram\-to\-reaction parsing\. Specifically, FG\-VLM is designed to \(i\) detect and recognize rigid chemical substructures \(functional groups\) and residual atoms inII, \(ii\) reconstruct their connectivity to form a topology\-faithful molecular graph, and \(iii\) recover the full reaction structure to derive canonical text representations \(*e\.g\.*, SMILES\)\. The detailed prompt specifications and instruction templates for each stage are provided in Appendix[A1](https://arxiv.org/html/2605.17214#A1)\. #### 3\.1\.1\.Reaction Diagram Deconstruction\. The first bottleneck in processing reaction diagrams is disentangling crowded visual elements\. As shown in the top panel of Figure[2](https://arxiv.org/html/2605.17214#S3.F2), we employ FG\-VLM as a layout parser to performReaction Deconstruction\. Given the raw imageII, the model detects bounding boxes for all semantic constituents:𝒪=\{\(ci,𝐛i\)\}i=1N\\mathcal\{O\}=\\\{\(c\_\{i\},\\mathbf\{b\}\_\{i\}\)\\\}\_\{i=1\}^\{N\}, whereNNis the number of objects and classesci∈\{Reactant,Product,Condition,Arrow,Plus\}c\_\{i\}\\in\\\{\\text\{Reactant\},\\allowbreak\\text\{Product\},\\allowbreak\\text\{Condition\},\\allowbreak\\text\{Arrow\},\\allowbreak\\text\{Plus\}\\\}\. Unlike generic object detectors, our model is trained to implicitly understand chemical syntax\. Based on the detected boxes𝐛i\\mathbf\{b\}\_\{i\}, we perform aCropoperation to generate a set of isolated molecular sub\-images, denoted asℐcrops=\{Icrop\(i\)∣ci∈\{Reactant,Product\}\}\\mathcal\{I\}\_\{crops\}=\\\{I\_\{crop\}^\{\(i\)\}\\mid c\_\{i\}\\in\\\{\\text\{Reactant\},\\text\{Product\}\\\}\\\}\. These isolated views serve as the input for the subsequent fine\-grained structure recognition\. #### 3\.1\.2\.Hybrid\-Granularity Visual Anchor Modeling\. For each cropped molecule imageIcrop∈ℐcropsI\_\{crop\}\\in\\mathcal\{I\}\_\{crops\}, we aim to reduce atomic redundancy while preserving exact topology\. We represent the molecule as a*hybrid\-granularity graph*G′=\(Vatom∪Vsuper,E′\)G^\{\\prime\}=\(V\_\{\\text\{atom\}\}\\cup V\_\{\\text\{super\}\},E^\{\\prime\}\), whereVsuperV\_\{\\text\{super\}\}denotes predefined semantic super\-nodes \(*e\.g\.*, Phenyl, Carboxyl\), including terminal groups as well as bridging/scaffold groups with multiple attachment sites, andVatomV\_\{\\text\{atom\}\}contains the remaining residual atoms\. The edge setE′E^\{\\prime\}encodes bonds among all nodes inVatom∪VsuperV\_\{\\text\{atom\}\}\\cup V\_\{\\text\{super\}\}, including Atom–Atom, Atom–SuperNode, and SuperNode–SuperNode bonds\. ##### Hybrid\-Granularity Nodes and Bonds\. GivenIcropI\_\{crop\}, FG\-VLM predicts a unified node setV=Vatom∪VsuperV=V\_\{\\text\{atom\}\}\\cup V\_\{\\text\{super\}\}and a bond setEE\. Each super\-nodeg∈Vsuperg\\in V\_\{\\text\{super\}\}is represented as\(idg,nameg,𝐛g\)\(id\_\{g\},name\_\{g\},\\mathbf\{b\}\_\{g\}\), whereidgid\_\{g\}is a unique node ID,namegname\_\{g\}specifies the functional\-group type, and𝐛g=\(xg,1,yg,1,xg,2,yg,2\)\\mathbf\{b\}\_\{g\}=\(x\_\{g,1\},y\_\{g,1\},x\_\{g,2\},y\_\{g,2\}\)is its bounding box\. Each residual atoma∈Vatoma\\in V\_\{\\text\{atom\}\}is represented as\(ida,elema,𝐛a\)\(id\_\{a\},elem\_\{a\},\\mathbf\{b\}\_\{a\}\), whereelema∈\{C,N,O,…\}elem\_\{a\}\\in\\\{\\mathrm\{C,N,O,\\dots\}\\\}\. Each bond is represented as\(u,v,o\)∈E\(u,v,o\)\\in E, whereuuandvvare node IDs \(atom or super\-node\) andoois the bond order\. We use string IDs to distinguish node types: atom nodes use the prefixA\_\(*e\.g\.*,A\_1\), and functional\-group nodes use the prefixFG\_\(*e\.g\.*,FG\_1\); IDs are unique within each molecule crop\. ##### Visual Anchors\. Detecting a functional group thus requires more than bounding boxes: the model must infer which internal attachment sites of a rigid group participate in external bonds\. For a group instanceg∈Vsuperg\\in V\_\{\\text\{super\}\}, we define itsvisual anchor set𝒜g=\{𝐱\(1\),…,𝐱\(kg\)\}\\mathcal\{A\}\_\{g\}=\\\{\\mathbf\{x\}^\{\(1\)\},\\ldots,\\mathbf\{x\}^\{\(k\_\{g\}\)\}\\\}, where each anchor𝐱\(j\)=\(x,y\)\\mathbf\{x\}^\{\(j\)\}=\(x,y\)is the pixel coordinate of an internal atom inggthat forms a bond with the external structure \(*i\.e\.*, a residual atom or another super\-node\)\.kg=\|𝒜g\|k\_\{g\}=\|\\mathcal\{A\}\_\{g\}\|equals11for terminal groups andkg≥2k\_\{g\}\\geq 2for bridging/scaffold groups\. In a subsequent step, we crop a group\-specific imageIgI\_\{g\}from the molecule crop using the predicted bounding box𝐛g\\mathbf\{b\}\_\{g\}, and reuse FG\-VLM in a second pass onIgI\_\{g\}to output the anchor set𝒜g\\mathcal\{A\}\_\{g\}\. ##### Serialized Supervision\. We serialize the hybrid structure prediction \(nodes and bonds\) as a target sequenceS=SV⊕SES=S\_\{V\}\\oplus S\_\{E\}, whereSVS\_\{V\}enumerates all nodes andSES\_\{E\}enumerates all bonds: \(1\)SV\\displaystyle S\_\{V\}=⨁g∈Vsuper\{idg,nameg,𝐛g\}⊕⨁a∈Vatom\{ida,elema,𝐛a\},\\displaystyle=\\bigoplus\_\{g\\in V\_\{\\text\{super\}\}\}\\\{id\_\{g\},name\_\{g\},\\mathbf\{b\}\_\{g\}\\\}\\oplus\\ \\bigoplus\_\{a\\in V\_\{\\text\{atom\}\}\}\\\{id\_\{a\},elem\_\{a\},\\mathbf\{b\}\_\{a\}\\\},SE\\displaystyle S\_\{E\}=⨁\(u,v,o\)∈E\{u,v,o\}\.\\displaystyle=\\bigoplus\_\{\(u,v,o\)\\in E\}\\\{u,v,o\\\}\. #### 3\.1\.3\.Structural Reconstruction via Directional Vector Matching\. Given the predicted nodes and bonds\(V,E\)\(V,E\)and the anchor set𝒜g\\mathcal\{A\}\_\{g\}for each rigid groupg∈Vsuperg\\in V\_\{\\text\{super\}\}, reconstructingG′G^\{\\prime\}requires orientingggand mapping its anchors to discrete attachment atoms of its class template\. A naive alternative is to predict an explicit continuous pose parameter for each group \(*e\.g\.*, a rotation angle\) and then rotate the template accordingly\. However, such parameterization is brittle under periodicity and rotational symmetries\. We instead adopt a purely geometric procedure:Directional Vector Matching\. LetTcgT\_\{c\_\{g\}\}be the canonical 2D template of group classcgc\_\{g\}centered at the origin, withNcgN\_\{c\_\{g\}\}candidate attachment atoms\. We denote by𝐮cg\(i\)∈ℝ2\\mathbf\{u\}\_\{c\_\{g\}\}^\{\(i\)\}\\in\\mathbb\{R\}^\{2\}the template direction vector from the template center to theii\-th candidate attachment atom \(i∈\{1,…,Ncg\}i\\in\\\{1,\\ldots,N\_\{c\_\{g\}\}\\\}\)\. For each predicted anchor𝐱\(j\)∈𝒜g\\mathbf\{x\}^\{\(j\)\}\\in\\mathcal\{A\}\_\{g\}, we compute its direction vector relative to the predicted bounding box center ofgg, where𝐛g=\(xg,1,yg,1,xg,2,yg,2\)\\mathbf\{b\}\_\{g\}=\(x\_\{g,1\},y\_\{g,1\},x\_\{g,2\},y\_\{g,2\}\)and𝐜box=\(xg,1\+xg,22,yg,1\+yg,22\)\\mathbf\{c\}\_\{box\}=\\big\(\\frac\{x\_\{g,1\}\+x\_\{g,2\}\}\{2\},\\frac\{y\_\{g,1\}\+y\_\{g,2\}\}\{2\}\\big\): \(2\)𝐯pred\(j\)=𝐱\(j\)−𝐜box\.\\mathbf\{v\}\_\{pred\}^\{\(j\)\}=\\mathbf\{x\}^\{\(j\)\}\-\\mathbf\{c\}\_\{box\}\.We map each anchorjjto a template index by maximizing cosine similarity: \(3\)mj∗=argmaxi∈\{1,…,Ncg\}\(𝐯pred\(j\)⋅𝐮cg\(i\)‖𝐯pred\(j\)‖‖𝐮cg\(i\)‖\)\.m^\{\*\}\_\{j\}=\\arg\\max\_\{i\\in\\\{1,\\ldots,N\_\{c\_\{g\}\}\\\}\}\\left\(\\frac\{\\mathbf\{v\}\_\{pred\}^\{\(j\)\}\\cdot\\mathbf\{u\}\_\{c\_\{g\}\}^\{\(i\)\}\}\{\\\|\\mathbf\{v\}\_\{pred\}^\{\(j\)\}\\\|\\\|\\mathbf\{u\}\_\{c\_\{g\}\}^\{\(i\)\}\\\|\}\\right\)\.This discretizes rotation estimation into a matching problem\. For symmetric groups \(*e\.g\.*,para\-phenylene\), independent matching resolves rotational ambiguity by explicitly mapping each anchor to a specific canonical atom index on the template \(*e\.g\.*, distinguishing the opposing C1 and C4 attachment sites on a benzene ring\)\. Once correspondences are obtained, we align the template to the visual input and instantiate the oriented group inG′G^\{\\prime\}by attaching its incident predicted bonds according to the matched anchors\. #### 3\.1\.4\.Reaction Reconstruction\. Following directional vector matching, we instantiate the complete molecular graph by placing the oriented functional\-group templates and attaching the incident predicted bonds to the matched attachment sites, together with the predicted residual atoms\. We convert the reconstructed graph into a RDKit\(Landrum and others,[2013](https://arxiv.org/html/2605.17214#bib.bib38)\)molecule and export a canonical SMILES string, using RDKit sanitization to validate valence and connectivity\. Finally, in theReaction Reconstructionphase, we reassemble the global reaction equation \(*e\.g\.*, Reactants→\\toProducts\) by organizing the individual SMILES strings according to the spatial layout parsed in Section[3\.1\.1](https://arxiv.org/html/2605.17214#S3.SS1.SSS1), thereby producing a structured reaction representation\. ### 3\.2\.Semantic Activation While FG\-VLM provides topology\-faithful SMILES and a structured reaction representation, downstream LLMs can still suffer from a semantic disconnect when prompted with SMILES alone\. We hypothesize that LLMs index scientific knowledge more effectively through natural\-language entity names than through structural strings\(Zhanget al\.,[2024a](https://arxiv.org/html/2605.17214#bib.bib43)\)\. As a post\-recognition enhancement, we introduce aSemantic Activationpipeline that converts the recognition output into semantically enriched prompts\. ##### Entity Resolution\. We employ a deterministic mapping functionℳ:𝒵→ℰ\\mathcal\{M\}:\\mathcal\{Z\}\\to\\mathcal\{E\}that converts the recognized SMILES stringZZinto a set of entity namesℰ\\mathcal\{E\}by querying the corresponding compound record in PubChem\(Kimet al\.,[2023](https://arxiv.org/html/2605.17214#bib.bib37)\)\. Specifically, we retrieve the IUPAC name and the synonym list provided by PubChem, and form two name strings:nIUPACn\_\{\\text\{IUPAC\}\}\(the retrieved IUPAC name\) andncommonn\_\{\\text\{common\}\}\(a comma\-separated list of selected synonyms after de\-duplication\)\.We use both common names and IUPAC names as they are complementary: common names are frequent in natural\-language chemistry texts and better align with LLM pre\-training, while IUPAC names provide a standardized, broadly available identifier for long\-tail molecules\. ##### Activation Prompting\. We construct an enriched promptPactP\_\{act\}that integrates the visual description with the resolved entities: \(4\)Pact=\\displaystyle P\_\{act\}="Image shows \[ncommon,nIUPAC\]\\displaystyle\\texttt\{"Image shows \[\}n\_\{\\text\{common\}\},n\_\{\\text\{IUPAC\}\}\\texttt\{\]\}\(SMILES: \[Z\]\)\. Analyze\.\.\."\\displaystyle\\texttt\{ \(SMILES: \[\}Z\\texttt\{\]\)\. Analyze\.\.\."\}We leverage these name\-enhanced prompts to better elicit the model’s parametric chemical knowledge, such as reactivity rules, electronic effects, and synthesis priors\. Table 1\.Main Results on OCRD\-Bench\.Comparison between the Baseline \(Direct Input\) and our ChemVA Framework \(Ours\)\.Rec: Recognition,Know: Knowledge,Reas: Reasoning\. ChemVA shows consistent double\-digit improvements across all LLMs\.ModelsVanilla LLMLLM\+ChemVAGainRecKnowReasTotalRecKnowReasTotalProprietary SOTA ModelsGemini 2\.5 Pro68\.059\.054\.058\.8092\.074\.071\.076\.40\+17\.60GPT\-5 \(Preview\)64\.053\.050\.554\.2092\.069\.066\.272\.48\+18\.28Qwen3\-Max61\.050\.048\.051\.4092\.070\.065\.772\.68\+21\.28GPT\-4o45\.048\.046\.546\.8092\.067\.062\.570\.20\+23\.40Claude 4 Sonnet49\.047\.045\.246\.6892\.066\.062\.569\.80\+23\.12Open\-WeightsYi\-Lightning47\.041\.038\.741\.2892\.058\.054\.763\.48\+22\.20Llama\-4\-109B44\.038\.536\.738\.8892\.055\.052\.061\.20\+22\.32Intern\-S154\.043\.541\.244\.6892\.056\.553\.562\.40\+17\.72ChemVLM36\.027\.525\.728\.4892\.032\.530\.543\.60\+15\.12 ### 3\.3\.Data Construction and Training for FG\-VLM This subsection describes how we construct supervision and train FG\-VLM to acquire the hybrid\-granularity perception and visual\-anchor capabilities introduced in Section[3\.1](https://arxiv.org/html/2605.17214#S3.SS1)\. #### 3\.3\.1\.Dataset Construction Training FG\-VLM requires explicit annotations for functional\-group boxes, residual\-atom boxes, bond connectivity \(node IDs and bond orders\), and visual anchors, which are not available in standard atom\-level OCSR datasets\. To this end, we buildFG\-SFT, a specialized instruction\-tuning dataset for hybrid\-granularity recognition\. FG\-SFT contains 100k molecule samples and 10k reaction samples; detailed statistics and source distributions are provided in Table[A1](https://arxiv.org/html/2605.17214#A2.T1)\(Appendix[A2](https://arxiv.org/html/2605.17214#A2)\)\. ##### Molecule\-Anchor Subset\. We source unique molecular structures from PubChem\(Kimet al\.,[2023](https://arxiv.org/html/2605.17214#bib.bib37)\)\. For each molecule, we start from its\.molfile and construct the atom–bond graph with bond orders\. We then apply a priority\-driven greedy decomposition \(Appendix[A2](https://arxiv.org/html/2605.17214#A2)\) to partition atoms into functional\-group instances\{FGi\}\\\{FG\_\{i\}\\\}; atoms not assigned to anyFGiFG\_\{i\}are treated as residual atoms\. This yields the hybrid\-granularity node set \(functional\-group nodes \+ residual\-atom nodes\) together with the bond set inherited from the\.molconnectivity\. The atom\-wise exclusivity constraint in the decomposition enforces a bijective mapping between each functional\-group token and its underlying atom set\. To derive visual anchors, we inspect cross\-boundary bonds between the atom set ofFGiFG\_\{i\}and atoms outsideFGiFG\_\{i\}\(either residual atoms or atoms in other functional groups\)\. The endpoint atoms*inside*FGiFG\_\{i\}that participate in such bonds are defined as its attachment atoms and used as anchors\. We render each molecule into a 2D diagram with RDKit and obtain anchor keypoints by mapping these attachment atoms to their 2D coordinates in the depiction \(converted to pixel coordinates under the drawing canvas\), providing the supervision for anchor prediction\. ##### Reaction\-Layout Subset\. We leverage the Open Reaction Database \(ORD\)\(Kearneset al\.,[2021](https://arxiv.org/html/2605.17214#bib.bib39)\)to source chemically valid reaction records\. To approximate the layouts seen in literature, we synthesize reaction diagrams in three archetypes:Linear Flow\(70%\),Multi\-line\(15%\), andTree/Graph\(15%\); all diagram constituents are automatically annotated with bounding boxes\. #### 3\.3\.2\.Training Details We fine\-tune a single FG\-VLM \(Qwen2\.5\-VL\-32B\)\(Baiet al\.,[2025](https://arxiv.org/html/2605.17214#bib.bib79)\)to perform all perception stages in a unified instruction\-following, sequence\-generation formulation\. Each training example contains a visual inputXX\(a full reaction diagramII, a molecule cropIcropI\_\{\\text\{crop\}\}, or a functional\-group cropIgI\_\{g\}\), a task instruction, and a structured target sequenceYY\(layout boxes, hybrid nodes\+bonds, or anchors\)\. ##### Supervised Objective\. We optimize the teacher\-forced negative log\-likelihood: \(5\)ℒNLL\(X,Y\)=−∑t=1\|Y\|logpθ\(yt∣y<t,X\),\\mathcal\{L\}\_\{\\text\{NLL\}\}\(X,Y\)=\-\\sum\_\{t=1\}^\{\|Y\|\}\\log p\_\{\\theta\}\\\!\\left\(y\_\{t\}\\mid y\_\{<t\},X\\right\),whereyty\_\{t\}is thett\-th output token in the serialized target\. ##### Task Definitions\. We train FG\-VLM with three instruction formats: \(i\)Reaction DeconstructiononII: predict diagram constituents and bounding boxes \(Section[3\.1\.1](https://arxiv.org/html/2605.17214#S3.SS1.SSS1)\); \(ii\)Hybrid Structure PredictiononIcropI\_\{\\text\{crop\}\}: predict the serialized hybrid graph sequenceS=SV⊕SES=S\_\{V\}\\oplus S\_\{E\}\(Section[3\.1\.2](https://arxiv.org/html/2605.17214#S3.SS1.SSS2)\); \(iii\)Anchor PredictiononIgI\_\{g\}: predict the anchor list𝒜g=\{𝐱\(j\)\}j=1kg\\mathcal\{A\}\_\{g\}=\\\{\\mathbf\{x\}^\{\(j\)\}\\\}\_\{j=1\}^\{k\_\{g\}\}for each functional group \(Section[3\.1\.2](https://arxiv.org/html/2605.17214#S3.SS1.SSS2)\)\. We denote the corresponding losses asℒlayout\\mathcal\{L\}\_\{\\text\{layout\}\},ℒgraph\\mathcal\{L\}\_\{\\text\{graph\}\}, andℒanchor\\mathcal\{L\}\_\{\\text\{anchor\}\}\. ##### Three\-Stage Curriculum\. We adopt a curriculum\-style schedule that progressively introduces finer\-grained supervision: - •Stage I \(Graph pretraining\)\. We first train FG\-VLM on the molecule subset \(100k molecules\) to learn hybrid structure prediction onIcropI\_\{\\text\{crop\}\}, optimizingℒgraph\\mathcal\{L\}\_\{\\text\{graph\}\}only\. - •Stage II \(Anchor refinement\)\. We then add anchor supervision on functional\-group cropsIgI\_\{g\}from the same molecules, jointly optimizingℒgraph\+ℒanchor\\mathcal\{L\}\_\{\\text\{graph\}\}\+\\mathcal\{L\}\_\{\\text\{anchor\}\}to improve fine\-grained attachment perception while retaining hybrid graph prediction ability\. - •Stage III \(Joint instruction tuning\)\. Finally, we mix reaction\-layout samples \(10k reactions\) with the molecule\-level graph and anchor samples, and optimize a weighted multi\-task objective: \(6\)ℒ=\\displaystyle\\mathcal\{L\}\\;=λlayoutℒlayout\+λgraphℒgraph\+λanchorℒanchor\.\\displaystyle\\lambda\_\{\\text\{layout\}\}\\mathcal\{L\}\_\{\\text\{layout\}\}\+\\lambda\_\{\\text\{graph\}\}\\mathcal\{L\}\_\{\\text\{graph\}\}\+\\lambda\_\{\\text\{anchor\}\}\\mathcal\{L\}\_\{\\text\{anchor\}\}\. In Stage III, mini\-batches are drawn from the three task pools with fixed sampling probabilities \(treated as hyperparameters\), while all tasks share the same FG\-VLM parameters across input granularities\. ##### Cropping Strategy\. During training, molecule cropsIcropI\_\{\\text\{crop\}\}and group cropsIgI\_\{g\}are generated from ground\-truth bounding boxes to avoid compounding errors; at inference, they are generated from FG\-VLM predictions\. ## 4\.Experiments ### 4\.1\.Experimental Setup ##### Implementation Details\. For the Hybrid\-Granularity Visual Module, we use Qwen2\.5\-VL\-32B as the perception backbone and fine\-tune it on our FG\-SFT dataset to support functional\-group grounding and visual anchor detection\. For the Semantic Activation stage, we query the evaluation LLMs with structured outputs \(SMILES and entity names\) using a standardized prompt that enforces a “Name\-First” reasoning trajectory: “Identify the reaction type based on the provided chemical name, then analyze the mechanism…”\. All hyperparameters \(*e\.g\.*, learning rates, batch sizes\) and the complete prompt templates are provided in Appendix\. ##### Baselines\. We benchmark ChemVA against a comprehensive suite of multimodal LLMs, grouped into proprietary SOTA models and open\-weights models \(Table[1](https://arxiv.org/html/2605.17214#S3.T1)\)\. In theDirect Inputbaseline, we supply the raw reaction image to each model and rely on its native visual encoder, which serves as a lower bound for end\-to\-end performance\. ##### Evaluation Dataset\. To rigorously evaluate diagram\-grounded reasoning, we introduceOCRD\-Bench, a challenging benchmark curated from graduate\-level chemistry assessment materials\. Unlike prior datasets centered on either isolated structure recognition or multiple\-choice answering, OCRD\-Bench emphasizes deep, multi\-step logical deduction\. Specifically, OCRD\-Bench contains 100 core reaction scenarios expanded into 500 hierarchical questions, spanning eight major categories that cover the pillars of organic chemistry \(see the taxonomy in Appendix Figure[A2](https://arxiv.org/html/2605.17214#A3.F2)\)\. We organize evaluation into three cognitive tiers that mirror the scientific reasoning pipeline: L1 \(Visual Perception\) measures grounding of visual evidence into symbolic representations; L2 \(Chemical Knowledge\) assesses retrieval of static reactivity rules and reaction conditions; and L3 \(Mechanistic Reasoning\) requires generating step\-by\-step electron\-flow explanations and key intermediates \(Figure[A3](https://arxiv.org/html/2605.17214#A3.F3)\)\. To ensure high\-quality ground truth, two PhD\-level domain experts independently verified and corrected all standard keys to ensure scientific accuracy and support reliable automatic evaluation\. ##### Evaluation Metric\. We employ task\-specific metrics to enable a rigorous assessment\. For L1, we report strict binary accuracy based on the Tanimoto similarity of Morgan fingerprints\. For L2 and L3, accuracy alone is insufficient: L2 uses multiple\-choice questions where therationaleis critical, whereas L3 is open\-ended\. We therefore adopt the LLM\-as\-a\-Judge paradigm\(Zhenget al\.,[2023](https://arxiv.org/html/2605.17214#bib.bib49)\), using Gemini 2\.5 Pro to assess both option correctness and the logical validity of the accompanying explanation against the reference answers\. Formal metric definitions and judge prompts are provided in Appendix[A3](https://arxiv.org/html/2605.17214#A3)\. Table 2\.Scale Robustness on Qwen2\.5\-VL\. Vnilla LLM vs\. ChemVA across different parameter scales within the same model family\. ChemVA yields stable gains, indicating low sensitivity to backbone size\. ### 4\.2\.Main Results Table[1](https://arxiv.org/html/2605.17214#S3.T1)compares the performance of the proposed ChemVA framework with that of the Vanilla LLM baseline\. ##### Overcoming the Visual Deficit\. The baseline results strongly validate the “Visual Deficit” hypothesis\. Even leading proprietary models struggle to interpret the topological semantics of molecular graphs\. While Gemini\-2\.5\-Pro achieves a recognition score of 68%, other large\-scale models such as GPT\-4o perform poorly, attaining only 45% accuracy in structural identification via direct input\. This observation suggests that general\-purpose vision encoders constitute the primary bottleneck\. Specialized models such as ChemVLM also exhibit limitations, with a Recognition score of 36%, which restricts their effectiveness in this domain\. In contrast, our ChemVA framework explicitly models functional groups and anchors, elevating the Recognition score to 92% across all backbones\. This structural correction provides a consistent and accurate foundation for downstream tasks\. ##### Performance Leap in Knowledge and Reasoning\. Beyond addressing visual perception, the proposed framework achieves substantial gains in downstream reasoning tasks \(refer to the Know & Reas columns in Table[1](https://arxiv.org/html/2605.17214#S3.T1)\)\. Our ChemVA framework significantly enhances the Total Score across all architectures\. This improvement is particularly pronounced for Open\-Weights models: the Total Score of Llama\-4\-109B increases from 38\.88% to 61\.2% \(\+22\.32\), while that of Yi\-Lightning improves from 41\.28% to 63\.48% \(\+22\.2\)\. Crucially, this comprehensive improvement enables open\-weights models to compete with proprietary SOTA systems\. With the proposed framework, Llama\-4\-109B \(61\.2%\) outperforms the direct input baseline of Gemini 2\.5 Pro \(58\.8%\)\. This empirically demonstrates that providing structured, high\-quality inputs that incorporate both precise visual graphs and semantic entities is critical for realizing the reasoning potential of these models\. ### 4\.3\.Robustness to Backbone Scale While Table[1](https://arxiv.org/html/2605.17214#S3.T1)compares a diverse set of architectures, variations across model families may obscure the specific effects of parameter scaling\. To isolate the impact of backbone size, we conduct a controlled scaling study using the Qwen2\.5\-VL family, with parameter counts ranging from 3B to 72B\. Each model is evaluated under two configurations: \(i\)Vanilla LLM, in which the model generates responses directly from raw reaction images via its native vision encoder; and \(ii\)LLM\+ChemVA, in which the model relies on topology\-faithful structures and entity names generated by FG\-VLM and the Semantic Activation pipeline\. As presented in Table[2](https://arxiv.org/html/2605.17214#S4.T2), ChemVA achieves a consistent absolute improvement ranging from \+25\.82 to \+27\.94 points in the Total score across all model sizes\. Although the performance of the Direct Input baseline improves with scale \(increasing from 6\.42 to 29\.84\), it remains significantly constrained by visual perception capabilities \(with Rec≤\\leq45% even at the 72B scale\)\. Conversely, ChemVA elevates the recognition score to 92\.0% for all backbones and consistently enhances performance in both the Knowledge and Mechanistic Reasoning tasks\. These results indicate that the primary bottleneck is unreliable visual grounding rather than insufficient linguistic capacity\. Consequently, employing topology\-faithful recognition combined with semantic activation offers an efficient alternative to simply increasing parameter scale\. ### 4\.4\.Ablation Studies To rigorously validate the effectiveness of our framework components, we conduct ablation studies focusing on two critical aspects: the impact of incorporating semantic entities \(Semantic Activation\) and the necessity of high\-fidelity structure recognition \(FG\-VLM\)\. ##### Bridging the Semantic Disconnect\. For qualitative case studies and detailed performance metrics, please refer to Appendix[A4](https://arxiv.org/html/2605.17214#A4)\. As illustrated in Figure[3](https://arxiv.org/html/2605.17214#S4.F3), the results demonstrate thatSemantic Activationserves as an effective solution to the semantic disconnect\. By incorporating Entity Names, consistent performance gains are observed across the entire spectrum of models\. This effect is particularly substantial in Open\-Weights models, where improvements exceed 10%\. For instance, the score of Llama\-4\-109B improves from a baseline of 48\.88% \(see Appendix Table[A2](https://arxiv.org/html/2605.17214#A3.T2)\) to 61\.2% \(\+12\.3%\), and that of Yi\-Lightning increases from 52\.08% to 63\.5% \(\+11\.4%\)\. This consistent enhancement confirms that ChemVA effectively bridges the semantic gap, enabling diverse architectures to accurately align visual representations with latent chemical reasoning\. ##### Replacing the Recognition Frontend\. To further verify that the observed gains are driven by accurate structure recognition, the FG\-VLM module is replaced with a range of off\-the\-shelf recognizers\. These include transformer\-based OCSR tools \(*e\.g\.*, RxnScribe\(Qianet al\.,[2023a](https://arxiv.org/html/2605.17214#bib.bib40)\)and DECIMER\(Schüttet al\.,[2020](https://arxiv.org/html/2605.17214#bib.bib11); Rajanet al\.,[2023](https://arxiv.org/html/2605.17214#bib.bib24)\)\) and instruction\-tuned chemical VLMs \(*e\.g\.*, ChemVLM\(Liet al\.,[2024](https://arxiv.org/html/2605.17214#bib.bib42)\)\)\. Table[3](https://arxiv.org/html/2605.17214#S4.T3)presents the L1 recognition accuracies on OCRD\-Bench\. While transformer\-based OCSR baselines reach 83–86%, VLM\-based recognizers exhibit severe hallucinations \(25–36%\), highlighting the limitations of generic encoders\. In contrast, the proposed FG\-VLM achieves 92%, surpassing all alternatives\. Furthermore, the most competitive external baseline \(RxnScribe\) is integrated into the pipeline to evaluate end\-to\-end performance across all nine LLM backbones\. As depicted in Figure[4](https://arxiv.org/html/2605.17214#S4.F4)and Appendix Table[A3](https://arxiv.org/html/2605.17214#A3.T3), the recognition ceiling of RxnScribe \(86%\) constrains the final Total Scores \(*e\.g\.*, Llama\-4\-109B: 57\.44 with RxnScribe vs\. 61\.20 with FG\-VLM\), demonstrating that topology\-faithful recognition is a prerequisite for reliable downstream reasoning\. Table 3\.Recognizer Substitution on OCRD\-Bench \(L1\)\.We compare several structure recognizers as drop\-in replacements for FG\-VLM\.Model TypeModel NameOCRD\-Bench \(L1 only\)TransformerRxnScribe86%Decimer83%VLMChemDFM\-X\-13B25%ChemVLM\-26B36%OursFG\-VLM92% Figure 3\.Effectiveness of Semantic Activation\. The stacked bar chart illustrates the Total Scores across all models\. The gray base represents the performance withSMILES Only, while the green top section highlights the Gain from Semantic Activation\. The contrast clearly shows that gains are universal, with all models demonstrating significant improvement\.Figure 4\.Superiority of FG\-VLM over RxnScribe\. The chart illustrates the Total Score comparison across 9 models\. The gray bars represent the performance using RxnScribe as the recognition tool, while the red sections indicate the explicit performance gains achieved by our FG\-VLM\. ## 5\.Conclusion In this work, we present a unified framework for chemical visual reasoning that bridges the critical gaps of visual deficit and semantic disconnect via hybrid\-granularity anchoring and semantic activation\. Built upon this framework, we develop a specialized visual module \(FG\-VLM\) and a semantic alignment pipeline, capable of precise functional group grounding and effective activation of latent chemical knowledge in Multimodal LLMs\. Moreover, we contribute two high\-quality resources to the community: FG\-SFT, a large\-scale hybrid\-granularity dataset for fine\-grained visual instruction tuning, and OCRD\-Bench, a challenging evaluation benchmark derived from graduate\-level exams requiring end\-to\-end mechanistic reasoning\. Extensive experiments have demonstrated that ChemVA acts as a powerful performance equalizer, enabling open\-weights models to surpass proprietary SOTA systems and achieving superior structural recognition accuracy compared to specialized tools\. ## Limitations and Ethical Considerations While our ChemVA framework improves chemical reasoning, two limitations remain: TheVisual Anchor Mechanismrelies on a predefined dictionary of functional groups\. While our current vocabulary covers over 95% of common substructures \(*e\.g\.*, in PubChem\), the model must revert to atom\-level prediction for rare or atypical groups outside this set\. This fallback loses the geometric robustness provided by the anchor vectors, potentially reducing recognition accuracy for exotic molecules\. In addition, theSemantic Alignmentmodule depends on mapping recognized structures \(SMILES\) to meaningful entity names\. For novel molecules or intermediates that lack established common names or database entries, the "Knowledge Activation" pipeline cannot trigger the LLM’s latent semantic networks\. In addition, this work focuses on chemical reaction diagram understanding for research purposes, and does not involve human subjects, personal data, or deployment in safety\-critical settings; we therefore do not anticipate specific ethical concerns beyond standard dataset licensing and responsible use of models\. ## References - N\. Alampara, I\. Mandal, P\. Khetarpal, H\. S\. Grover,et al\.\(2024\)MaCBench: a multimodal chemistry and materials science benchmark\.InNeurIPS 2024 Workshop AI for Materials,Cited by:[§2\.2](https://arxiv.org/html/2605.17214#S2.SS2.p1.1)\. - S\. Bai, K\. Chen, X\. Liu, J\. Wang, W\. Ge, S\. Song, K\. Dang, P\. Wang, S\. Wang, J\. Tang,et al\.\(2025\)Qwen2\. 5\-vl technical report\.arXiv preprint arXiv:2502\.13923\.Cited by:[§3\.3\.2](https://arxiv.org/html/2605.17214#S3.SS3.SSS2.p1.5)\. - \[3\]\(2025\)Benchmarking mllms on topological reasoning of chemical reaction diagrams\.OpenReview/arXiv Submission 1142\.Cited by:[§2\.2](https://arxiv.org/html/2605.17214#S2.SS2.p1.1)\. - D\. A\. Boiko, R\. MacKnight, G\. Gomes,et al\.\(2023\)Autonomous chemical research with large language models\.Nature624\(7992\),pp\. 570–578\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p1.1)\. - A\. M\. Branet al\.\(2025\)Chemical reasoning in llms unlocks steerable synthesis planning and reaction mechanism elucidation\.arXiv preprint arXiv:2503\.08537\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p1.1)\. - K\. Chen, Y\. Du, J\. Li, H\. Cao, M\. Guo, X\. Dang, L\. Li, J\. Qiu, G\. Chen, and P\. A\. Heng \(2025\)ChemMiner: a large language model agent system for chemical literature data mining\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 7595–7603\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p2.1)\. - D\. Clevert, T\. Le, R\. Winter, and F\. Montanari \(2021\)Img2Mol \- accurate molecular structure estimation from images\.Chemical Science12\(42\),pp\. 14174–14181\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p2.1)\. - Y\. Diaoet al\.\(2023\)MacFrag: segmenting large\-scale molecules to obtain diverse fragments\.Bioinformatics39\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p3.2)\. - C\. Edwardset al\.\(2022\)Translation between molecules and natural language\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p3.2)\. - \[10\]\(2025\)Evaluating the accuracy and educational potential of generative ai models in pharmacy education: a comparative analysis of chatgpt and gemini across bloom’s taxonomy\.Pharmacy\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p1.1)\. - V\. Fan, Y\. Qian, A\. Wang, A\. Wang, C\. W\. Coley, and R\. Barzilay \(2024\)OpenChemIE: an information extraction toolkit for chemistry literature\.Journal of Chemical Information and Modeling64\(14\),pp\. 5521–5534\.Cited by:[§2\.1](https://arxiv.org/html/2605.17214#S2.SS1.p2.1)\. - F\. Gobetet al\.\(2001\)Chunking mechanisms in human learning\.Trends in Cognitive Sciences5\(6\),pp\. 236–243\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p4.1)\. - Y\. Gu and Z\. Liang \(2025\)MolRAG: unlocking the power of llms for molecular property prediction\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(ACL 2025\),Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p1.1)\. - T\. Guo, K\. Guo, B\. Nan, Z\. Liang, Z\. Guo, N\. V\. Chawla, O\. Wiest, and X\. Zhang \(2023\)What can large language models do in chemistry? a comprehensive benchmark on eight tasks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.36\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p1.1)\. - S\. R\. Heller, A\. McNaught, I\. Pletnev, S\. Stein, and D\. Tchekhovskoi \(2015\)InChI, the iupac international chemical identifier\.Journal of cheminformatics7\(1\),pp\. 23\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p3.2)\. - S\. M\. Kearnes, M\. R\. Maser, M\. Wleklinski, A\. Kast, A\. G\. Doyle, S\. D\. Dreher, J\. M\. Hawkins, K\. F\. Jensen, and C\. W\. Coley \(2021\)The open reaction database\.Journal of the American Chemical Society143\(45\),pp\. 18820–18826\.Cited by:[§A2\.1](https://arxiv.org/html/2605.17214#A2.SS1.p2.1),[§3\.3\.1](https://arxiv.org/html/2605.17214#S3.SS3.SSS1.Px2.p1.1)\. - S\. Kim, J\. Chen, T\. Cheng, A\. Gindulyte, J\. He, S\. He, Q\. Li, B\. A\. Shoemaker, P\. A\. Thiessen, B\. Yu,et al\.\(2023\)PubChem 2023 update\.Nucleic Acids Research51\(D1\),pp\. D1373–D1380\.Cited by:[§3\.2](https://arxiv.org/html/2605.17214#S3.SS2.SSS0.Px1.p1.5),[§3\.3\.1](https://arxiv.org/html/2605.17214#S3.SS3.SSS1.Px1.p1.5)\. - G\. Landrumet al\.\(2013\)RDKit: open\-source cheminformatics\.Note:[http://www\.rdkit\.org](http://www.rdkit.org/)Accessed: 2025\-05\-20Cited by:[§A2\.2](https://arxiv.org/html/2605.17214#A2.SS2.SSS0.Px2.p1.1),[§3\.1\.4](https://arxiv.org/html/2605.17214#S3.SS1.SSS4.p1.1)\. - J\. Li, D\. Zhang, X\. Wang, Z\. Hao,et al\.\(2025\)Chemvlm: exploring the power of multimodal large language models in chemistry area\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.39,pp\. 415–423\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p1.1)\. - J\. Li, D\. Zhang, D\. Zhou,et al\.\(2024\)ChemVLM: exploring the power of multimodal large language models in chemistry area\.arXiv preprint arXiv:2408\.07246\.Cited by:[§2\.1](https://arxiv.org/html/2605.17214#S2.SS1.p3.1),[§2\.2](https://arxiv.org/html/2605.17214#S2.SS2.p1.1),[§4\.4](https://arxiv.org/html/2605.17214#S4.SS4.SSS0.Px2.p1.1)\. - T\. Lin, M\. Maire, S\. Belongie, J\. Hays, P\. Perona, D\. Ramanan, P\. Dollár, and C\. L\. Zitnick \(2014\)Microsoft coco: common objects in context\.InEuropean conference on computer vision,pp\. 740–755\.Cited by:[§A2\.1](https://arxiv.org/html/2605.17214#A2.SS1.p3.pic1.1.1.1.1.1.1.1)\. - P\. Lu, S\. Mishra, T\. Xia, L\. Qiu, K\. Chang, S\. Zhu, O\. Tafjord, P\. Clark, and A\. Kalyan \(2022\)Learn to explain: multimodal reasoning via thought chains for science question answering\.InThe 36th Conference on Neural Information Processing Systems \(NeurIPS\),Cited by:[§2\.2](https://arxiv.org/html/2605.17214#S2.SS2.p1.1)\. - \[23\]\(2025\)MolecularIQ: characterizing chemical reasoning capabilities through symbolic verification on molecular graphs\.Under Review at ICLR 2026\.Cited by:[§2\.2](https://arxiv.org/html/2605.17214#S2.SS2.p1.1)\. - L\. Morin, M\. Danelljan, M\. I\. Agea,et al\.\(2023\)MolGrapher: graph\-based visual recognition of chemical structures\.InProceedings of the IEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 19552–19561\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p2.1)\. - M\. Oldenhof, A\. Arany, Y\. Moreau, and J\. Simm \(2021\)Self\-labeling of fully mediating representations by graph alignment\.InBenelux Conference on Artificial Intelligence,pp\. 46–65\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.17214#S2.SS2.p1.1)\. - Y\. Qian, J\. Guo, R\. Barzilay, and C\. Coley \(2023a\)RxnScribe: a sequence generation model for reaction diagram parsing\.InJournal of Chemical Information and Modeling,Vol\.63,pp\. 4030–4041\.Cited by:[§4\.4](https://arxiv.org/html/2605.17214#S4.SS4.SSS0.Px2.p1.1)\. - Y\. Qian, J\. Guo, Z\. Tu, Z\. Li, C\. W\. Coley, and R\. Barzilay \(2023b\)MolScribe: robust molecular structure recognition with image\-to\-graph generation\.Journal of Chemical Information and Modeling63\(18\),pp\. 5833–5844\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.17214#S2.SS1.p2.1),[§3\.1](https://arxiv.org/html/2605.17214#S3.SS1.p1.2)\. - Y\. Qian, J\. Guo, Z\. Tu, Z\. Li, C\. W\. Coley, and R\. Barzilay \(2024\)RxnScribe: a unified framework for chemical reaction diagram parsing\.arXiv preprint arXiv:2305\.11845\.Cited by:[§2\.1](https://arxiv.org/html/2605.17214#S2.SS1.p2.1)\. - K\. Rajan, H\. O\. Brinkhaus, M\. I\. Agea, A\. Zielesny, and C\. Steinbeck \(2023\)DECIMER\.ai: an open platform for automated optical chemical structure identification, segmentation and recognition in scientific publications\.Nature communications14\(1\),pp\. 5045\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.17214#S2.SS1.p2.1),[§4\.4](https://arxiv.org/html/2605.17214#S4.SS4.SSS0.Px2.p1.1)\. - L\. Research, S\. Chun, J\. Kim, A\. Jo, Y\. Jo, S\. Oh, S\. Lee, K\. Ryoo, J\. Lee, S\. H\. Kim,et al\.\(2025\)MolMole: molecule mining from scientific literature\.arXiv preprint arXiv:2505\.03777\.Cited by:[§2\.1](https://arxiv.org/html/2605.17214#S2.SS1.p3.1)\. - N\. T\. Runcieet al\.\(2025\)Can reasoning power significantly improve the knowledge of large language models for chemistry?\.Journal of Chemical Information and Modeling\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p3.2)\. - N\. T\. Runcie, C\. M\. Deane, and F\. Imrie \(2025\)ChemIQ: a benchmark for chemical reasoning and molecular comprehension\.arXiv preprint arXiv:2505\.07735\.Cited by:[§2\.2](https://arxiv.org/html/2605.17214#S2.SS2.p1.1)\. - C\. Schütt, K\. Rajan, A\. Zielesny, and C\. Steinbeck \(2020\)DECIMER 1\.0: deep learning for chemical image recognition using transformers\.Journal of Chemical Information and Modeling60,pp\. 5359–5372\.Note:Also published in J\. Cheminf\. as separate work, please verify specific citationCited by:[§1](https://arxiv.org/html/2605.17214#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.17214#S2.SS1.p2.1),[§4\.4](https://arxiv.org/html/2605.17214#S4.SS4.SSS0.Px2.p1.1)\. - A\. K\. Shah, A\. Dey, L\. Luo, B\. Amador, P\. Philippy, M\. Zhong, S\. Ouyang, D\. M\. Friday, D\. Bianchi, N\. Jackson,et al\.\(2025\)Multimodal search in chemical documents and reactions\.InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval,pp\. 4030–4034\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p2.1)\. - J\. Staker, K\. Marshall, R\. Abel, and C\. M\. McQuaw \(2019\)Molecular structure extraction from documents using deep learning\.Journal of chemical information and modeling59\(3\),pp\. 1017–1029\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p2.1)\. - J\. Wang, H\. Yang, J\. Wu, Y\. He, X\. Wei, Y\. Wang, C\. Liu, L\. Ge, L\. Wu, B\. Wang,et al\.\(2025\)GTR\-cot: graph traversal as visual chain of thought for molecular structure recognition\.arXiv preprint arXiv:2506\.07553\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p2.1)\. - X\. Wang, Y\. Zhu, Z\. Liu,et al\.\(2024\)SciBench: evaluating college\-level scientific problem\-solving abilities of large language models\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§2\.2](https://arxiv.org/html/2605.17214#S2.SS2.p1.1)\. - D\. M\. Wilary and J\. M\. Cole \(2021\)ReactionDataExtractor: a tool for automated extraction of information from chemical reaction schemes\.Journal of chemical information and modeling61\(10\),pp\. 4962–4974\.Cited by:[§2\.1](https://arxiv.org/html/2605.17214#S2.SS1.p2.1)\. - D\. M\. Wilary and J\. M\. Cole \(2023\)ReactionDataExtractor 2\.0: a deep learning approach for data extraction from chemical reaction schemes\.Journal of Chemical Information and Modeling63\(19\),pp\. 6053–6067\.Cited by:[§2\.1](https://arxiv.org/html/2605.17214#S2.SS1.p2.1)\. - R\. Xu, Y\. Zhang, Q\. Wang, C\. Edwards, and H\. Ji \(2025\)OMeBench: towards robust benchmarking of llms in organic mechanism elucidation and reasoning\.Note:arXiv preprint arXiv:2510\.07731Cited by:[§2\.2](https://arxiv.org/html/2605.17214#S2.SS2.p1.1)\. - Z\. Yu, X\. Xu, and H\. Gao \(2024\)G2t\-llm: graph\-to\-tree text encoding for molecule generation with fine\-tuned large language models\.arXiv preprint arXiv:2410\.02198\.Cited by:[§2\.1](https://arxiv.org/html/2605.17214#S2.SS1.p2.1)\. - X\. Yue, Y\. Ni, K\. Zhang, T\. Zheng, R\. Liu, G\. Zhang, S\. Stevens, D\. Jiang, W\. Ren, Y\. Sun,et al\.\(2024\)Mmmu: a massive multi\-discipline multimodal understanding and reasoning benchmark for expert agi\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 9556–9567\.Cited by:[§2\.2](https://arxiv.org/html/2605.17214#S2.SS2.p1.1)\. - D\. Zhang, W\. Liu,et al\.\(2024a\)ChemLLM: a chemical large language model\.arXiv preprint arXiv:2402\.06852\.Cited by:[§2\.2](https://arxiv.org/html/2605.17214#S2.SS2.p1.1),[§3\.2](https://arxiv.org/html/2605.17214#S3.SS2.p1.1)\. - S\. Zhang, W\. Liu,et al\.\(2024b\)Igniting the power of large language models for chemistry: a systematic survey\.arXiv preprint arXiv:2401\.14656\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p1.1)\. - L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing,et al\.\(2023\)Judging llm\-as\-a\-judge with mt\-bench and chatbot arena\.arXiv preprint arXiv:2306\.05685\.Cited by:[§4\.1](https://arxiv.org/html/2605.17214#S4.SS1.SSS0.Px4.p1.1)\. - J\. Zhuang, K\. Li, J\. Hou, M\. Xu, Z\. Gao, and H\. Cai \(2025\)Doc2SAR: a synergistic framework for high\-fidelity extraction of structure\-activity relationships from scientific documents\.arXiv preprint arXiv:2506\.21625\.Cited by:[§1](https://arxiv.org/html/2605.17214#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.17214#S2.SS1.p3.1)\. ## Appendix A1Implementation Details: Prompts and Instruction Tuning ### A1\.1\.Reaction Diagram Deconstruction Prompt To initiate the pipeline, we employ the following prompt to decompose the raw reaction image into semantic components\. Beyond simple object detection, this prompt enforces alogic\-aware grouping mechanismvia thereaction\_idfield\. This design is critical for handling complex scenarios such as multi\-step synthesis \(*e\.g\.*,A→B→CA\\to B\\to C\) or disconnected reaction variants, ensuring that shared intermediates are correctly associated with their respective reaction steps\. The structured JSON output containing bounding boxes is subsequently used to crop individual molecular images for fine\-grained recognition\. Reaction Parsing PromptSystem PromptYou are an expert chemical reaction diagram analysis assistant\. Your task is to analyze chemical reaction diagrams and parse them into structured component lists\. You must accurately detect all visual elements, classify their roles, and group them by reaction pathways\. Handle complex layouts including linear flows, multi\-step synthesis, and divergent/convergent pathways\. Respond strictly in the specified JSON format\.User Prompt\#\#\# Task Description \#\#\#Analyze the provided chemical reaction diagram image and extract all semantic components\. For each detected object, provide: 1\.Reaction ID: An integer identifier grouping components belonging to the same reaction step\. If multiple distinct reactions are present \(*e\.g\.*, Step 1 and Step 2\), assign distinct IDs\. Shared components \(*e\.g\.*, an intermediate serving as product of R1 and reactant of R2\) should be listed foreachreaction they participate in\. 2\.Role: Classify the object into one of: "Reactant", "Product", "Condition", "Arrow", "Plus"\. 3\.BBox: The bounding box coordinates \[x1, y1, x2, y2\] normalized to 0\-1000\.\#\#\# Parsing Rules \#\#\#Role Definitions:\-Reactant: Starting materials or substrates appearing before the reaction arrow\.\-Product: Final compounds or intermediates appearing after the reaction arrow\.\-Condition: Text or small formulas above/below arrows \(reagents, solvents, temperature, time\)\.\-Arrow: The graphical arrow indicating reaction direction\.\-Plus: The "\+" sign separating multiple reactants or products\.Grouping Logic:\- A "Reaction" consists of a set of Reactants \+ Conditions \+ Arrow→\\toProducts\. \- In multi\-step reactions \(A→B→CA\\to B\\to C\), "B" is the Product of Reaction 1 and the Reactant of Reaction 2\. \- In disconnected layouts \(multiple independent reactions in one image\), treat them as separate Reaction IDs\.\#\#\# Output Format \(STRICT\) \#\#\#Respond with a valid JSON list of objects\. Do not include markdown code blocks \(“‘json\)\. \[ \{"reaction\_id": 1, "role": "Reactant", "bbox": \[x1, y1, x2, y2\]\}, \{"reaction\_id": 1, "role": "Arrow", "bbox": \[x1, y1, x2, y2\]\}, \{"reaction\_id": 1, "role": "Condition", "bbox": \[x1, y1, x2, y2\]\}, \{"reaction\_id": 1, "role": "Product", "bbox": \[x1, y1, x2, y2\]\}, \{"reaction\_id": 2, "role": "Reactant", "bbox": \[x1, y1, x2, y2\]\}, … \]\#\#\# Input Image \#\#\#\{input\_image\}Now begin your structured response: ### A1\.2\.Hybrid\-Granularity Structure Recognition Prompt This prompt drives the core hybrid\-granularity recognition\. The prompt enforces aTop\-Down Visual Perceptionstrategy\. We instruct the model to prioritize the detection of holistic functional group patterns based on the provided Priority List , treating atoms merely as residual elements to fill the gaps\. This "Chunking" strategy ensures topological robustness against visual noise\. Hybrid\-Granularity Structure RecognitionSystem PromptYou are an expert chemical structure recognition assistant\. Your task is to convert a cropped molecule image into aHybrid\-Granularity Graph\. You must employ aTop\-Down Visual Perceptionstrategy: 1\. First, identify large, rigidSuper\-nodes\(Functional Groups\) based on global visual patterns\. 2\. Second, identifyIsolated Atomsonly in the remaining empty regions\. 3\. Finally, determine the connectivity \(Bonds\) between all identified nodes\.User Prompt\#\#\# Task Description \#\#\#Analyze the molecular image and generate a structured JSON representation containingsupernodes,atoms, andbonds\.\#\#\# Decomposition Constraints \(CRITICAL\) \#\#\#1\.Visual Priority \(Top\-Down\): \-Rule: Prioritize the detection of Functional Group Patterns \(from \{priority\_list\_file\}\) over individual atoms\. \-Strategy: Treat the image primarily as a composition of Functional Groups\. Identify these rigid structures based on their global visual features\. \-Conflict Resolution: If a visual region matches a Functional Group template, you MUST output it as a Super\-node\. Do NOT decompose it into internal atoms\.2\.Reference Priority List: Strict adherence to the provided file:\{priority\_list\_file\}\. \-Greedy Matching: Scan and match functional groups strictly in the order they appear in the file\. \-Closed\-Set Constraint: Only recognize Super\-nodes explicitly listed in \{priority\_list\_file\}\. If a substructure is not in the list, treat it as distinct Isolated Atoms\.3\.Residual Atom Identification: \-Rule: Only scan forIsolated Atomsin the image regions that areNOTcovered by the bounding boxes of identified Super\-nodes\. \-Logic: Atoms are "residual" elements\. If an atom is visually encapsulated within a Super\-node’s region, it is implicitly handled by the Super\-node template and must not be listed here\.4\.Super\-node Exclusivity \(No Overlap\): \-Rule: Super\-nodes must NOT share any common atoms\. \-Logic: While bounding boxes of adjacent groups may spatially overlap due to crowding, the underlying atomic membership is mutually exclusive\. No single atom can belong to more than one Super\-node\. If ambiguity exists, assign ownership to the group appearing earlier in the Priority List\. \#\#\# Output Schema Definitions \#\#\#1\.SuperNodes: \-id: Unique identifier \(*e\.g\.*, "fg\_1"\)\. \-label: Must match a name from \{priority\_list\_file\}\. \-bbox: \[x1, y1, x2, y2\]\.2\.Atoms\(Residual\): \-id: Unique identifier \(*e\.g\.*, "atom\_1"\)\. \-symbol: Element symbol \(*e\.g\.*, "C", "N", "O"\)\. \-bbox: \[x1, y1, x2, y2\]\.3\.Bonds: \-source: Node ID \(Atom or SuperNode\)\. \-target: Node ID \(Atom or SuperNode\)\. \-type: Bond order \("SINGLE", "DOUBLE", "TRIPLE", "AROMATIC", "WEDGE", "DASH"\)\. \*Note: Connects any pair of nodes \(Atom\-Atom, SuperNode\-SuperNode, or Atom\-SuperNode\)\.\*\#\#\# Output Format \(STRICT\) \#\#\#Respond with a single valid JSON object\.\{"supernodes": \[ \.\.\. \],"atoms": \[ \.\.\. \],"bonds": \[ \.\.\. \]\}\#\#\# Input Image \#\#\#\{cropped\_molecule\_image\}\#\#\# Functional Group Priority List \(File X\) \#\#\#\{priority\_list\_content\}Now begin your analysis, prioritizing visual patterns from the list: ### A1\.3\.Fine\-Grained Visual Anchor Prediction Prompt While Phase 2 establishes the topological connectivity, it lacks the precise geometric alignment required for reconstructing the chemical drawing\. To bridge this gap, Phase 3 employs a specializedVisual Grounding Prompt\. This prompt takes the bounding box and class label of a detected super\-node \(from Phase 2\) as input and queries the VLM for the precise coordinates of itsVisual Anchors—the specific atoms within the group that serve as attachment points to the external skeleton\. These coordinates are the prerequisites for the subsequentDirectional Vector Matchingalgorithm\. Prompt for Phase 3: Visual Anchor PredictionSystem PromptYou are a fine\-grained visual grounding assistant for chemical structures\. Your task is to identify the precise "Visual Anchors" within a specific functional group\.Definition: A Visual Anchor is an atombelonging to the functional groupthat forms a chemical bond with an atomoutsidethe group\. \-Terminal Groups\(*e\.g\.*, \-OH, \-Cl\): Typically have 1 anchor\. \-Linker Groups\(*e\.g\.*, Phenyl, Pyridine\): Can have 2 or more anchors depending on the substitution pattern \(para\-, meta\-, ortho\-\)\.User Prompt\#\#\# Task Description \#\#\#Focus exclusively on the functional group located at the bounding box:\{target\_bbox\}\. The class of this group is:\{target\_label\}\.Your goal is to find the attachment points to the external molecule skeleton\. 1\. Observe the specified region\. 2\. Identify the specific atom\(s\) inside this group that connect to external bonds\. 3\. Return the center coordinates \[x, y\] of these anchor atoms \(normalized 0\-1000\)\.\#\#\# Output Format \(STRICT\) \#\#\#Respond with a JSON object containing a single list of coordinates\.\{"anchors": \[\[x1, y1\],\[x2, y2\]\]\}\#\#\# Input Image \#\#\#\{cropped\_molecule\_image\}Now, locate the anchors for the\{target\_label\}at\{target\_bbox\}: ## Appendix A2Data Construction Details ### A2\.1\.Image Synthesis Configuration Standard datasets typically offer binary modalities: either high\-level text captions \(*e\.g\.*, molecule names\) or low\-level dense coordinates \(*e\.g\.*, pure “\.mol” files\)\. Neither format is sufficient to teach a model the expert\-like cognitive process of identifying functional groups\. To address this, we constructedFG\-SFT, a specialized dataset designed to instill hybrid\-granularity perception into the model\. Unlike simple Image\-SMILES pairs, FG\-SFT provides a structured annotation layer that sits between raw atoms and semantic entities\. It comprises 100k Molecule samples and 10k Reaction samples, where each entry explicitly maps visual regions to functional group and defines their specificattachment points\(anchors\)\. This unique format compels the model to learn both the semantic identity of chemical substructures and their precise geometric connectivity within the global reaction context\. To ensure visual robustness, we synthesized images using 8 distinct templates, varying parameters such as bond thickness, font style, and atom label visibility\. For reaction diagrams, we leveraged the Open Reaction Database \(ORD\)\(Kearneset al\.,[2021](https://arxiv.org/html/2605.17214#bib.bib39)\)\. Table A1\.Statistics of the FG\-SFT Training Dataset\.The dataset is balanced across molecular recognition and reaction reasoning tasks\.Note:†\\dagger: Sourced from ORD/Open\-Source Datasets\.‡\\ddagger: Sourced from PubChem/Open\-Source Datasets\. Figure A1\.FG\-SFT example\.A raw reaction diagram with our FG\-SFT annotations, including component\-level boxes \(Reactants, Products, Conditions, Arrow, Plus\) and fine\-grained labels \(functional\-group boxes, bond boxes, and anchor keypoints\)\.FG\-SFT COCO\-style Annotation\(Linet al\.,[2014](https://arxiv.org/html/2605.17214#bib.bib78)\)\(Example\)[⬇](data:text/plain;base64,ewogICJpbWFnZXMiOiBbCiAgICB7ICJpZCI6IDAsICJ3aWR0aCI6IDQzOCwgImhlaWdodCI6IDE0OSB9LAogICAgeyAiaWQiOiAxLCAid2lkdGgiOiA5NCwgICJoZWlnaHQiOiA5NCAgfSwKICAgIHsgImlkIjogMiwgIndpZHRoIjogMjYsICAiaGVpZ2h0IjogMjIgIH0sCiAgICB7ICJpZCI6IDMsICJ3aWR0aCI6IDgyLCAgImhlaWdodCI6IDY2ICB9LAogICAgeyAiaWQiOiA0LCAid2lkdGgiOiA0NSwgICJoZWlnaHQiOiA0MyAgfQogIF0sCiAgImNhdGVnb3JpZXMiOiBbCiAgICB7ICJpZCI6IDAsICJuYW1lIjogIkFycm93IiB9LAogICAgeyAiaWQiOiAxLCAibmFtZSI6ICJDb25kaXRpb25zIiB9LAogICAgeyAiaWQiOiAyLCAibmFtZSI6ICJGdW5jdGlvbmFsR3JvdXAiIH0sCiAgICB7ICJpZCI6IDMsICJuYW1lIjogIlBsdXMiIH0sCiAgICB7ICJpZCI6IDQsICJuYW1lIjogIlByb2R1Y3RzIiB9LAogICAgeyAiaWQiOiA1LCAibmFtZSI6ICJSZWFjdGFudHMiIH0sCiAgICB7ICJpZCI6IDYsICJuYW1lIjogIkJvbmQiIH0sCiAgICB7ICJpZCI6IDcsICJuYW1lIjogIkFuY2hvciIgfQogIF0sCiAgImFubm90YXRpb25zIjogWwogICAgeyAiaWQiOiAwLCAiaW1hZ2VfaWQiOiAwLCAiY2F0ZWdvcnlfaWQiOiA1LCAiYmJveCI6IFs5Ljg0MjY5NjQyNjcxMzMxNCwgMjQuMTE0Njk5NDMzMjE2NjQsIDEyMy4wMzM3MDgxNjg5MTg3LCAxMDIuODU2MDI4MDk4MDQ1OTVdIH0sCiAgICB7ICJpZCI6IDEsICJpbWFnZV9pZCI6IDAsICJjYXRlZ29yeV9pZCI6IDUsICJiYm94IjogWzE1NC4wMzgyMDIyNDcxOTEwNCwgNjYuOTMwMzM3MDc4NjUxNjksIDMzLjk1NzMwMzM3MDc4NjU1LCAyOS41MjgwODk4ODc2NDA0NTVdIH0sCiAgICB7ICJpZCI6IDIsICJpbWFnZV9pZCI6IDAsICJjYXRlZ29yeV9pZCI6IDQsICJiYm94IjogWzI0NS4wODMxNDU4NjQ5MTU2LCAyNi41NzUyODA4OTg4NzYzOTMsIDEwMy44NDA0NDk2NDA3MDI1NywgOTAuNTUyNzgzNzA5MjI0ODhdIH0sCiAgICB7ICJpZCI6IDMsICJpbWFnZV9pZCI6IDAsICJjYXRlZ29yeV9pZCI6IDAsICJiYm94IjogWzE5My45MDExMjM1OTU1MDU2MiwgNzIuODM1OTU1MDU2MTc5NzgsIDQ1LjI3NjQwNDQ5NDM4MjAzNSwgMTUuMjU2MTc5Nzc1MjgwODk1XSB9LAogICAgeyAiaWQiOiA0LCAiaW1hZ2VfaWQiOiAwLCAiY2F0ZWdvcnlfaWQiOiA0LCAiYmJveCI6IFszNjguNjA4OTg4NzY0MDQ1MSwgNTEuNjc0MTkxMDA5NDIyOTksIDYxLjAyNDcxOTEwMTEyMzU0NCwgNTUuMTE5MDY3NDE3NTQzMjU2XSB9LAogICAgeyAiaWQiOiA1LCAiaW1hZ2VfaWQiOiAwLCAiY2F0ZWdvcnlfaWQiOiAzLCAiYmJveCI6IFsxMzYuODEzNDgzMTQ2MDY3NDUsIDcyLjgzNTk1NTA1NjE3OTc4LCAxMi4zMDMzNzA3ODY1MTY4MzgsIDE2LjI0MDQ0OTQzODIwMjI0NF0gfSwKCiAgICB7ICJpZCI6IDYsICJpbWFnZV9pZCI6IDEsICJjYXRlZ29yeV9pZCI6IDIsICJiYm94IjogWzQyLjc3MjkxNDQzMDgxODg0NCwgMTEuNjU0NjYyMDU1NDg4NjA4LCAzOC43MTg1NzY1NjI3NDc4MywgMzkuMTE1MDEwNTA4NTg3NjFdIH0sCiAgICB7ICJpZCI6IDcsICJpbWFnZV9pZCI6IDEsICJjYXRlZ29yeV9pZCI6IDIsICJiYm94IjogWzEuNjE3Njc2OTE5NzY5ODI5LCAyNy40ODg4NjEwODkzMDIzMiwgMzguNTg2NDM2NTExODU3OTI2LCAzNi4zMzk5NzI4ODc3MDkyMTZdIH0sCiAgICB7ICJpZCI6IDgsICJpbWFnZV9pZCI6IDEsICJjYXRlZ29yeV9pZCI6IDIsICJiYm94IjogWzQ2LjE1ODI3OTAwNzU0MzQwNiwgNjIuMDczNTQ2ODU4Mjc1NDUsIDQ1LjU5MDE0MTQzODUwMjIzLCAyMC4yMTgyMzk0NjExNjE4NThdIH0sCiAgICB7ICJpZCI6IDksICJpbWFnZV9pZCI6IDEsICJjYXRlZ29yeV9pZCI6IDYsICJiYm94IjogWzM2LjIyMTMzOTQyOTM1MjI1LCA0MS4wMzYzMzYyNDcxOTg2MDUsIDE0LjY2MzU5MjE1NDk3MDI1MywgMTIuMjU2MTMwMjA4OTgxMjY3XSB9LAogICAgeyAiaWQiOiAxMCwgImltYWdlX2lkIjogMSwgImNhdGVnb3J5X2lkIjogNiwgImJib3giOiBbNjEuNjA4ODY1OTQ4MjEzMTQsIDQ3LjcxMTIyODA2MDYxMzY1LCAxMC45NDI5NDM3NDcyODUwOTYsIDE3Ljk0NjUyNTgwMDA4ODcxNV0gfSwKICAgIHsgImlkIjogMTEsICJpbWFnZV9pZCI6IDEsICJjYXRlZ29yeV9pZCI6IDcsICJiYm94IjogWzY5LCA2NSwgMCwgMF0sICJrZXlwb2ludHMiOiBbNjksIDY1LCAyXSB9LAoKICAgIHsgImlkIjogMTIsICJpbWFnZV9pZCI6IDIsICJjYXRlZ29yeV9pZCI6IDIsICJiYm94IjogWzEuNDk4MjU5NTAyNDY3Mjg5OCwgNC42MzA4ODk4MzAxMzE1NDIsIDI0LjUwMTc0MDQ5NzUzMjY5LCAxNC42NjQ0ODQ0NjIwODMyMDddIH0sCgogICAgeyAiaWQiOiAxMywgImltYWdlX2lkIjogMywgImNhdGVnb3J5X2lkIjogMiwgImJib3giOiBbNDMuODMxMzUzNzEyNjQ3NjU1LCAxLjYyMzM3ODk3MzExNjQxNDUsIDM1LjcxNDM5MzE1MTk1MzM5NSwgMzcuNzQzNjU0NDcyOTY3MTNdIH0sCiAgICB7ICJpZCI6IDE0LCAiaW1hZ2VfaWQiOiAzLCAiY2F0ZWdvcnlfaWQiOiAyLCAiYmJveCI6IFs0NS40NTQ3MjM2NjYzNjkwODQsIDUxLjk0ODI1NTYxODcwNzUyLCAxOS40ODA1OTU4NTcwMTUzMSwgMTQuMDUxNzQ0MzgxMjkyNDg3XSB9LAogICAgeyAiaWQiOiAxNSwgImltYWdlX2lkIjogMywgImNhdGVnb3J5X2lkIjogMiwgImJib3giOiBbMi40MzUwOTQwMzM2NzE2OTQ2LCAyMS41MDk4MzY2MzcwMjU2NzUsIDM2LjUyNjA5NzY4MDM1ODkzLCA0MC41ODQ1NjE2NTM0Njg2Ml0gfSwKICAgIHsgImlkIjogMTYsICJpbWFnZV9pZCI6IDMsICJjYXRlZ29yeV9pZCI6IDYsICJiYm94IjogWzM1LjUzMTEzNDI2NjI3NzU3LCAyMy4wMTg4MjYwNzAyMjEzODMsIDE0Ljk5NTYzODUyNjc3NTk4NCwgMTEuNzQ4MTc1NjcyODMwNzM3XSB9LAogICAgeyAiaWQiOiAxNywgImltYWdlX2lkIjogMywgImNhdGVnb3J5X2lkIjogNiwgImJib3giOiBbMzUuOTEzMTI5NDQxMzk1NjMsIDQ5Ljk1MzcwODQ5MzI4MDU0LCAxMi41MTIzMjcwODczODMxMSwgOS4wNzM4MTEyNzUxMTEzNDZdIH0sCiAgICB7ICJpZCI6IDE4LCAiaW1hZ2VfaWQiOiAzLCAiY2F0ZWdvcnlfaWQiOiA3LCAiYmJveCI6IFs1MiwgMjQsIDAsIDBdLCAia2V5cG9pbnRzIjogWzUyLCAyNCwgMl0gfSwKCiAgICB7ICJpZCI6IDE5LCAiaW1hZ2VfaWQiOiA0LCAiY2F0ZWdvcnlfaWQiOiAyLCAiYmJveCI6IFszLjA5MDgwMzI1OTYwNDE5MTcsIDIuNDA5Nzc4ODEyNTcyNzU5LCA0MS45MDkxOTY3NDAzOTU4MSwgMzYuNjk0NTU3MzU4Njk0MDJdIH0KICBdLAogICJyZWxhdGlvbnMiOiBbXQp9)\{"images":\[\{"id":0,"width":438,"height":149\},\{"id":1,"width":94,"height":94\},\{"id":2,"width":26,"height":22\},\{"id":3,"width":82,"height":66\},\{"id":4,"width":45,"height":43\}\],"categories":\[\{"id":0,"name":"Arrow"\},\{"id":1,"name":"Conditions"\},\{"id":2,"name":"FunctionalGroup"\},\{"id":3,"name":"Plus"\},\{"id":4,"name":"Products"\},\{"id":5,"name":"Reactants"\},\{"id":6,"name":"Bond"\},\{"id":7,"name":"Anchor"\}\],"annotations":\[\{"id":0,"image\_id":0,"category\_id":5,"bbox":\[9\.842696426713314,24\.11469943321664,123\.0337081689187,102\.85602809804595\]\},\{"id":1,"image\_id":0,"category\_id":5,"bbox":\[154\.03820224719104,66\.93033707865169,33\.95730337078655,29\.528089887640455\]\},\{"id":2,"image\_id":0,"category\_id":4,"bbox":\[245\.0831458649156,26\.575280898876393,103\.84044964070257,90\.55278370922488\]\},\{"id":3,"image\_id":0,"category\_id":0,"bbox":\[193\.90112359550562,72\.83595505617978,45\.276404494382035,15\.256179775280895\]\},\{"id":4,"image\_id":0,"category\_id":4,"bbox":\[368\.6089887640451,51\.67419100942299,61\.024719101123544,55\.119067417543256\]\},\{"id":5,"image\_id":0,"category\_id":3,"bbox":\[136\.81348314606745,72\.83595505617978,12\.303370786516838,16\.240449438202244\]\},\{"id":6,"image\_id":1,"category\_id":2,"bbox":\[42\.772914430818844,11\.654662055488608,38\.71857656274783,39\.11501050858761\]\},\{"id":7,"image\_id":1,"category\_id":2,"bbox":\[1\.617676919769829,27\.48886108930232,38\.586436511857926,36\.339972887709216\]\},\{"id":8,"image\_id":1,"category\_id":2,"bbox":\[46\.158279007543406,62\.07354685827545,45\.59014143850223,20\.218239461161858\]\},\{"id":9,"image\_id":1,"category\_id":6,"bbox":\[36\.22133942935225,41\.036336247198605,14\.663592154970253,12\.256130208981267\]\},\{"id":10,"image\_id":1,"category\_id":6,"bbox":\[61\.60886594821314,47\.71122806061365,10\.942943747285096,17\.946525800088715\]\},\{"id":11,"image\_id":1,"category\_id":7,"bbox":\[69,65,0,0\],"keypoints":\[69,65,2\]\},\{"id":12,"image\_id":2,"category\_id":2,"bbox":\[1\.4982595024672898,4\.630889830131542,24\.50174049753269,14\.664484462083207\]\},\{"id":13,"image\_id":3,"category\_id":2,"bbox":\[43\.831353712647655,1\.6233789731164145,35\.714393151953395,37\.74365447296713\]\},\{"id":14,"image\_id":3,"category\_id":2,"bbox":\[45\.454723666369084,51\.94825561870752,19\.48059585701531,14\.051744381292487\]\},\{"id":15,"image\_id":3,"category\_id":2,"bbox":\[2\.4350940336716946,21\.509836637025675,36\.52609768035893,40\.58456165346862\]\},\{"id":16,"image\_id":3,"category\_id":6,"bbox":\[35\.53113426627757,23\.018826070221383,14\.995638526775984,11\.748175672830737\]\},\{"id":17,"image\_id":3,"category\_id":6,"bbox":\[35\.91312944139563,49\.95370849328054,12\.51232708738311,9\.073811275111346\]\},\{"id":18,"image\_id":3,"category\_id":7,"bbox":\[52,24,0,0\],"keypoints":\[52,24,2\]\},\{"id":19,"image\_id":4,"category\_id":2,"bbox":\[3\.0908032596041917,2\.409778812572759,41\.90919674039581,36\.69455735869402\]\}\],"relations":\[\]\} ### A2\.2\.Priority\-driven Greedy Decomposition Protocol We employ a specialized decomposition algorithm to resolve semantic ambiguity in functional group identification\. The protocol consists of three specific steps: ##### 1\. Priority Hierarchy Definition\. We constructed a hierarchical dictionary where functional groups are ranked by heavy atom count, topological complexity, and semantic weight\. High\-complexity groups \(*e\.g\.*, Carboxyl−COOH\-\\text\{COOH\}, Amide−CONH2\-\\text\{CONH\}\_\{2\}\) are assigned higher matching priority than their constituents \(*e\.g\.*, CarbonylC=OC=O, Hydroxyl−OH\-OH\)\. ##### 2\. Recursive Matching with Exclusivity\. For each SMILES string, we perform recursive substructure matching using RDKit\(Landrum and others,[2013](https://arxiv.org/html/2605.17214#bib.bib38)\)\. Crucially, we enforce anAtom\-wise Exclusivity Constraint: once an atom is assigned to a high\-priority "super\-node" \(*e\.g\.*, the Carbon in−COOH\-\\text\{COOH\}\), it is locked and explicitly excluded from subsequent scans\. This prevents the redundant labeling of a carboxyl carbon as a ketone, ensuring a bijective mapping between visual anchors and semantic tokens\. ##### 3\. Residual Atom Handling\. After the greedy matching process, any remaining atoms \(typically satisfying the saturated alkane skeleton\) are retained as atomic tokens\. Thishybrid\-granularityapproach ensures that the model captures both high\-level functional semantics and low\-level structural details\. We then calculate the 2D bounding box and the precise anchor coordinates\(xanc,yanc\)\(x\_\{anc\},y\_\{anc\}\)for each identified group\. priority\_list\_file:1\.Quaternary Ammonium2\.Carboxylate3\.Boronic Acid4\.Boronate Ester5\.Carboxylic Anhydride6\.Orthoester7\.Carbonate Ester8\.Lactone9\.Acetal/Ketal10\.Hemiacetal/Hemiketal11\.Epoxide12\.Imide13\.Lactam14\.Carbamate15\.Amidine16\.Carboxylic Acid17\.Ester18\.Amide19\.Acyl Halide20\.Carbothioic O\-Acid21\.Carbothioic S\-Acid22\.Carbodithioic Acid23\.Thiolester24\.Thionoester25\.Isocyanate26\.Isothiocyanate27\.Cyanate28\.Thiocyanate29\.Nitrile / Cyano30\.Isonitrile31\.Nitro32\.Nitroso33\.Oxime34\.Azide35\.Azo36\.Hydrazine37\.Primary Amine38\.Secondary Amine39\.Tertiary Amine40\.Imine41\.Sulfo \(Sulfonic Acid\)42\.Sulfonate43\.Sulfonyl44\.Sulfino45\.Disulfide46\.Thiol/Sulfhydryl47\.Sulfide\(Thioether\)48\.Sulfinyl\(Sulfoxide\)49\.Aldehyde50\.Acetyl51\.Ketone/Carbonyl52\.Phenol53\.Alcohol/Hydroxyl54\.Hydroperoxy55\.Peroxy56\.Ether57\.Methylenedioxy58\.Aryl Halide59\.Alkyl Halide60\.Alkynyl61\.Alkenyl62\.Pyridyl63\.Imidazole64\.Aryl65\.Carbonyl66\.Halo ## Appendix A3OCRD\-Bench Framework: Design and Metrics To comprehensively evaluate Multimodal Large Language Models \(MLLMs\) on organic chemistry reasoning, we designedOCRD\-Bench, a hierarchical benchmark covering 8 major reaction categories \(see Figure[A2](https://arxiv.org/html/2605.17214#A3.F2)\)\. The evaluation is structured into three cognitive tiers, ranging from visual perception to deep mechanistic reasoning\. Here, we detail the task design and the corresponding metric protocol for each level\. Figure A2\.Taxonomy of OCRD\-Bench\.The benchmark spans 8 major categories of organic reactions, ranging from fundamental Hydrocarbons to complex Transition Metal Catalyzed Reactions\. This diversity ensures a robust evaluation of model generalization across different chemical domains\.### A3\.1\.Level 1: Visual Perception \(L1\) Task Definition \(Q1–Q2\)\.This tier tests the fundamental ability to ground visual features to symbolic representations\. TakingFriedel\-Crafts Alkylationas an example, the model is prompted:"Identify the substrate \(Benzene\) and reagent \(Chloroethane\) in the image and encode them into SMILES\." Metric: Strict Structural Equivalence\.Standard text\-generation metrics \(e\.g\., Levenshtein distance\) fail to capture chemical validity\. Instead, we utilize RDKit to generate Morgan Fingerprints \(radius=2, 2048 bits\) for both the predicted \(PP\) and ground\-truth \(GG\) SMILES\. The score is calculated as a strict binary metric: \(A1\)SL1=𝟙\(Tanimoto\(P,G\)=1\.0\)×100\\text\{S\}\_\{L1\}=\\mathbb\{1\}\(\\text\{Tanimoto\}\(P,G\)=1\.0\)\\times 100where𝟙\\mathbb\{1\}is the indicator function\. This strict criterion ensures that any hallucinated atom, incorrect bond type, or stereochemical error results in a zero score, prioritizing topological fidelity over partial character matches\. ### A3\.2\.Level 2: Chemical Knowledge \(L2\) Task Definition \(Q3–Q4\)\.This tier assesses the retrieval of static knowledge regarding reactivity, selectivity \(regio\-/chemo\-\), and reaction conditions\. Example task:"Compare the reactivity of chloroethane vs\. chlorobenzene in this reaction system\. Explain the scope and limitations regarding carbocation rearrangement\." Metric: LLM\-as\-a\-Judge \(Knowledge with Rationale\)\.While L2 tasks follow a single\-choice or multiple\-choice format, we do not rely solely on option accuracy to prevent "lucky guesses\." We employ a strong proprietary model \(Gemini 2\.5 Pro\) as an impartial judge to assess two components: - •Option Correctness:Whether the selected answer matches the key\. - •Rationale Validity:Whether the generated explanation logically derives the answer using correct chemical principles\. A high score is awarded only when both the choice and the reasoning are correct\. ### A3\.3\.Level 3: Mechanistic Reasoning \(L3\) Task Definition \(Q5\)\.This tier tests deep reasoning capabilities\. The model must generate a valid causal chain of intermediates rather than simply hallucinating the final product\. Example task:"Provide the complete reaction mechanism, describing the electron flow and intermediate structures using SMILES sequences\." Metric: LLM\-as\-a\-Judge \(Completeness & Consistency\)\.For open\-ended mechanism generation, the judge compares the model’s response against the standard exam key based on: - •Completeness:Coverage of all key intermediates \(e\.g\., formation of the Wheland intermediate\) and electron flow steps\. - •Consistency:Logical flow and absence of chemically impossible steps\. The judge assigns a scalar consistency score \(0\-100\) reflecting the semantic alignment with the ground truth mechanism\. ### A3\.4\.Detailed Performance Data Table[A2](https://arxiv.org/html/2605.17214#A3.T2)provides the fine\-grained metrics for the baseline setting, serving as the reference point for the gains illustrated in the main text\. Table A2\.Detailed Breakdown of Baseline Performance \(SMILES Only\)\. This table lists the fine\-grained metrics \(Rec, Know, Reas\) for the visual\-only setting\. The specific gains achieving the final Total Scores are visualized in Figure[3](https://arxiv.org/html/2605.17214#S4.F3)\.Table A3\.Detailed Performance Benchmark with Rxnscribe\. This table lists the fine\-grained metrics \(Rec, Know, Reas\) when using the external SOTA tool \(RxnScribe\) as the visual frontend\. Note that the Recognition score is standardized to the tool’s performance ceiling \(86\.0%\)\.Figure A3\.An Example Case from OCRD\-Bench\.We use theSuzuki\-couplingto illustrate our hierarchical evaluation design\.L1 \(Q1, Q2\)assesses visual perception and structure grounding\.L2 \(Q3, Q4\)evaluates static knowledge retrieval\.L3 \(Q5\)tests deep mechanistic reasoning\. ## Appendix A4Case Study To demonstrate the efficacy of our method in handling complex chemical reasoning, we conduct a detailed qualitative analysis on a representative sample from the OCRD\-Bench \(Transition Metal Catalysis\)\. We select the Suzuki\-Miyaura Coupling reaction, a cornerstone of organic synthesis, which requires the model to navigate through a multi\-step reasoning chain: from structural recognition to condition extraction, and finally to mechanistic inference\. Through this case study, we aim to highlight two critical failure modes prevalent in traditional Multimodal Large Language Models \(MLLMs\): - •Visual Deficit:The inability to accurately resolve fine\-grained structural details \(e\.g\., bond orders, stereochemistry, and connectivity\) in dense molecular diagrams\. - •Semantic Disconnect:The representation\-knowledge mismatch\. Since LLMs acquire the vast majority of their chemical knowledge from textbooks—where entities are primarily referenced by names or aliases \(e\.g\., "Aryl halide"\) rather than linear strings—SMILES often fails to effectively "awaken" \(activate\) the model’s deep\-seated domain knowledge\. ### A4\.1\.Visual Deficit in General\-Purpose VLMs To explicitly demonstrate theVisual Deficitdiscussed in Section[1](https://arxiv.org/html/2605.17214#S1), we present a qualitative comparison using a Suzuki\-Miyaura coupling reaction sample from the OCRD\-Bench\. As shown in Figure[A3](https://arxiv.org/html/2605.17214#A3.F3), the input image contains both dense molecular graphs \(reactants/products\) and textual reaction conditions\. Vanilla LLM: Qwen3\-MaxContext:Single\-step Reaction Parsing PromptQ1, Q2 System Prompt:You are an expert organic chemist and an advanced Optical Chemical Structure Recognition \(OCSR\) system\. Your task is to accurately extract chemical information from reaction scheme images\.User Prompt:Please analyze the attached image of a chemical reaction\. Your goal is to identify the chemical structures and the text conditions, and extract them into a structured format\.Instructions:•Reactants:Identify the molecules to the left of the reaction arrow\. Convert their structures into SMILES\. If there are multiple reactants, separate them with a period \(\.\)\.•Products:Identify the molecules to the right of the reaction arrow\. Convert their structures into SMILES\.•Conditions:Read the text above and below the reaction arrow\. Based on your chemical knowledge, categorize these substances into "Reagents" \(including catalysts, bases, ligands, additives\) and "Solvents"\.Output Format:Please strictly follow this format \(do not output markdown code blocks, just the raw text\):Reactants: \[SMILES string\]Products: \[SMILES string\]Reagents: \[List of reagents, separated by comma\]Solvents: \[List of solvents, separated by comma\]Answer:Reactants: COC\(=O\)c1ccc\(c2cc\(Br\)ccc2c1\)c1ccccc1\.OB\(O\)\(O\)c1ccccc1 Products: COC\(=O\)c1ccc\(c2ccccc2c1\)c1ccccc1 Reagents: Pd\(OAc\)2, PPh3, TBAB Solvents: Toluene, H2O FG\-VLM AnswerAnswer:Reactants: COC\(=O\)c1ccc\(\-c2ccc\(Br\)cc2\)cc1\.OB\(O\)c1ccccc1 Products: COC\(=O\)c1ccc\(\-c2ccc\(\-c3ccccc3\)cc2\)cc1 Reagents: Pd\(OAc\)2, PPh3, TBAB Solvents: Toluene, H2O ##### The Illusion of Understanding\. General\-purpose VLMs, such as Qwen3\-max, exhibit a deceptive capability profile\. As observed in the detailed model outputs, Qwen3\-max successfully retrieves the textual reaction conditions, correctly identifying the catalyst system \(“Pd\(OAc\)2, PPh3”\) and solvents \(“Toluene, H2O”\)\. This aligns with the model’s strong OCR capabilities for standard text\. However, a critical failure occurs in the structural interpretation of the molecular components \(Q1\)\. The ground truth reactant is a biaryl bromide \(methyl 4’\-\(bromophenyl\)\-benzoate\), composed of two linearly connected benzene rings\. In contrast, Qwen3\-max generates a hallucinatory SMILES string: COC\(=O\)c1ccc\(c2cc\(Br\)ccc2c1\)c1ccccc1 This output reveals a severe topological error: it incorrectly introduces an additional phenyl ring \(“c1ccccc1”\) and disrupts the biaryl connectivity, effectively fabricating a molecule that does not exist in the image\. This exemplifies theVisual Deficit: the model perceives the visual tokens but fails to resolve the strict graph topology, resulting in structural hallucination\. ##### Resolving Connectivity via Visual Anchors\. In contrast, our FG\-vLM correctly reconstructs the reactant SMILES by leveraging the proposedHybrid\-Granularity Visual Anchor Modeling\. Instead of treating the molecule as a sequence of atomic characters, FG\-vLM identifies the rigid functional groups \(e\.g\., the phenyl rings and ester group\) as stable visual anchors\. By explicitly modeling the directional vectors between these anchors, our method enforces topological fidelity, successfully recognizing the correct two\-ring system and the precise position of the bromine substituent\. This case underscores that bridging the visual deficit requires more than scaling generic vision encoders; it demands domain\-specific grounding of chemical substructures\. ### A4\.2\.Beyond Correct Answers: Grounding Reasoning in Visual Reality While quantitative metrics \(Accuracy\) indicate the final performance, a qualitative inspection of the reasoning process reveals the profound impact of our ChemVA framework in bridging theSemantic Disconnect\. We analyze two representative knowledge tasks \(Q3 and Q4\) to demonstrate the difference betweenparametric retrievalandvisually grounded reasoning\. Know TaskContext:Prompt with Knowledge InjectionSystem Prompt:You are an expert organic chemist and academic researcher\. Your task is to analyze chemical reaction images and answer reasoning questions based on your domain knowledge of reaction mechanisms, reagents, and catalytic cycles\.CRITICAL OUTPUT RULES:\(1\)You must output the response STRICTLY in the following format\.\(2\)Do not include any introductory text or concluding remarks\.\(3\)For "answer": Output the single letter \(e\.g\., A\) or letters separated by commas\.\(4\)For "explanation": Provide a concise but professional chemical justification\.Standard Output Format:answer: \[Option Letter\(s\)\] explanation: \[Your detailed reasoning content\]User Prompt:Task:Analyze the provided chemical reaction image and the supplementary information, and then answer the following question\.Supplementary information:\[\]Question:\[\]Question Type:\[\]SMILES OnlySupplementary Information \(Input\):Reactant1:COC\(=O\)c1ccc\(\-c2ccc\(Br\)cc2\)cc1,Reagents2:OB\(O\)c1ccccc1,Product1:COC\(=O\)c1ccc\(\-c2ccc\(\-c3ccccc3\)cc2\)cc1,Reagents:Pd\(OAc\)2,PPh3,TBAB;solvents:Toluene,H2OQ3Model Output:answer: B explanation: The reaction depicted is a Suzuki\-Miyaura cross\-coupling between an aryl bromide and phenylboronic acid, catalyzed by Pd\(OAc\)2with PPh3as ligand and TBAB as phase\-transfer catalyst in a toluene/water mixture\. Although the base \(e\.g\., K2CO3\) is not explicitly shown in the diagram, it is essential for the catalytic cycle\. The primary role of the base is to deprotonate the boronic acid, forming a more nucleophilic boronate species \[R\-B\(OH\)3\]\-, which facilitates the transmetallation step—where the aryl group from the palladium complex transfers to the boronate, enabling coupling\. This step is crucial for efficient coupling\. Option A is incorrect because the reduction of Pd\(II\) to Pd\(0\) is typically achieved by the ligands or solvent, not the base\. Option C is incorrect because benzyne formation is not involved; this is not a elimination\-based mechanism\. Option D is incorrect because the base actively participates in the transmetallation step, not merely neutralizing acid\. Thus, B is correct\.Q4Model Output:answer: D explanation: The reaction shown is a Suzuki\-Miyaura cross\-coupling, which proceeds via a Pd\(0\)/Pd\(II\) catalytic cycle\. The correct chronological sequence of elementary steps is: \(1\) Oxidative Addition—where the Pd\(0\) species adds across the C\-\-Br bond of the aryl bromide to form a Pd\(II\) complex; \(2\) Transmetallation—where the organoboron reagent \(phenylboronic acid, activated by base\) transfers its phenyl group to the palladium center, displacing the halide; and \(3\) Reductive Elimination—where the two aryl groups on palladium couple to form the biaryl product and regenerate the Pd\(0\) catalyst\. This sequence is well\-established in palladium\-catalyzed cross\-coupling reactions\. Option A is incorrect because transmetallation cannot occur before oxidative addition, as no Pd\-\-C bond exists yet\. Option B reverses the order of transmetallation and reductive elimination, which is mechanistically invalid\. Option C starts with reductive elimination, which requires prior formation of the Pd\(II\) intermediate, making it impossible as the first step\. Thus, D is the only correct sequence\.ChemVASupplementary Information \(Input\):Reactant1:COC\(=O\)c1ccc\(\-c2ccc\(Br\)cc2\)cc1,Synonyms:methyl4’\-bromo\[1,1’\-biphenyl\]\-4\-carboxylate,IUPACName:methyl4\-\(4\-bromophenyl\)benzoate,Reagents2:OB\(O\)c1ccccc1,Synonyms:Phenylboronicacid,IUPACName:phenylboronicacid,Product1:COC\(=O\)c1ccc\(\-c2ccc\(\-c3ccccc3\)cc2\)cc1,Synonyms:,IUPACName:methyl4\-\(4\-phenylphenyl\)benzoate,Reagents:Pd\(OAc\)2,PPh3,TBAB;solvents:Toluene,H2OQ3Model Output:answer: B explanation: The reaction shown is a Suzuki–Miyaura cross\-coupling between methyl 4\-\(4\-bromophenyl\)benzoate and phenylboronic acid, catalyzed by Pd\(OAc\)2/PPh3in the presence of TBAB, under biphasic conditions \(toluene/H2O\)\. In this catalytic cycle, the base \(e\.g\., K2CO3\) plays a critical role during the transmetallation step\. It deprotonates the boronic acid to form a more nucleophilic boronate species \[R\-B\(OH\)3\]\-, which facilitates the transfer of the organic group from boron to palladium\. This step is essential because the neutral boronic acid is less reactive toward transmetallation with the Pd\(II\) or Pd\(0\) intermediate\. Option A is incorrect because the reduction of Pd\(II\) to Pd\(0\) is typically facilitated by phosphine ligands or solvent\-mediated processes, not the base\. Option C is incorrect as benzyne formation requires strong bases and specific substrates \(e\.g\., ortho\-dihalides\), which are not present here\. Option D is incorrect because the base is not merely a stoichiometric neutralizer; it actively participates in the mechanism via coordination and activation of the boronic acid\. Thus, the primary mechanistic role of the base is to generate the reactive boronate complex, making option B correct\.Q4Model Output:answer: D explanation: The reaction depicted is a Suzuki\-Miyaura cross\-coupling between an aryl bromide \(methyl 4\-\(4\-bromophenyl\)benzoate\) and phenylboronic acid, catalyzed by Pd\(OAc\)2in the presence of PPh3and TBAB in toluene/water\. This transformation proceeds via a well\-established Pd\(0\)/Pd\(II\) catalytic cycle\. The correct chronological sequence of elementary steps is: \(1\) Oxidative Addition, where the Pd\(0\) species adds across the C\-\-Br bond of the aryl halide to form a Pd\(II\) intermediate; \(2\) Transmetallation, in which the organoboron reagent transfers its phenyl group to the palladium center, displacing the bromide and forming a Pd\(II\)\-aryl complex; and \(3\) Reductive Elimination, where the two aryl groups coupled on the Pd\(II\) center eliminate to form the biaryl product and regenerate Pd\(0\)\. Option D correctly lists this sequence: Oxidative Addition→\\toTransmetallation→\\toReductive Elimination\. Options A, B, and C are incorrect because they misorder these essential mechanistic steps—transmetallation cannot occur before oxidative addition, and reductive elimination must follow transmetallation to form the new C\-\-C bond\. ##### Q3: Mechanistic Fidelity vs\. Hallucinated Logic\. In Question 3 \(Role of the Base\), both the baseline \(SMILES Only\) and ChemVA correctly selected Option B\. However, a close examination of the generated explanations reveals a critical divergence in chemical understanding\. - •The Baseline’s Failure: Although the baseline selected the correct option, its explanation contains a fundamental mechanistic error\. It states:“the aryl group from the palladium complex transfers to the boronate\.”This is chemically inverted; in Transmetallation, the organic group transfersfromthe activated boron speciestothe palladium center\. This “Right Answer, Wrong Reason” phenomenon suggests the baseline relies on superficial keyword matching or hallucinated logic rather than a true understanding of the catalytic cycle\. - •ChemVA’s Precision: In contrast, ChemVA accurately describes the directionality of the group transfer \(Boron→\\toPalladium\) and correctly identifies the nucleophilic activation process\. This confirms that our Visual Anchor mechanism not only aids in structure recognition but also enforces logical consistency in downstream reasoning\. ##### Q4: Instance\-Level Awareness vs\. Generic Templates\. Question 4 asks for the sequence of the catalytic cycle\. While the sequence is standard for all Suzuki couplings, the quality of the response depends on how well the model integrates the specific context of the input image\. - •Generic Retrieval \(Baseline\): The baseline refers to the reactant simply as an “aryl bromide\.” While factually true, this generic label implies the model is retrieving a pre\-trained template for “Suzuki Reaction” without attending to the specific molecular details shown in the image\. - •Contextual Grounding \(ChemVA\): ChemVA exhibits superior context awareness by explicitly naming the complex substrate:“methyl 4\-\(4\-bromophenyl\)benzoate\.”This precision stems directly from our Semantic Activation mechanism, which translates visual features into explicit textual anchors \(e\.g\., IUPAC names and synonyms\) before reasoning\. It also correctly enumerates the specific reaction conditions \(Pd\(OAc\)2, PPh3, TBAB\) visible in the image\. This proves that ChemVA is not merely reciting a textbook definition but is performinginstance\-specific reasoninggrounded in the visual evidence extracted via the Semantic Activation module\. Reas TaskContext:Open\-Ended Reasoning PromptSystem Prompt:You are an expert organic chemist and synthetic consultant\. Your task is to analyze chemical reaction images and answer advanced reasoning questions regarding substrate scope, reaction kinetics, and condition optimization\.CRITICAL OUTPUT RULES:\(1\)You must output the response STRICTLY in the following format\.\(2\)Do not include any introductory text\.\(3\)For "answer": Provide a concise, direct conclusion \(1\-2 sentences\)\.\(4\)For "explanation": Provide a comprehensive analysis including Theoretical Basis, Mechanism Analysis, and Synthetic Strategy\.Standard Output Format:answer: \[Direct Conclusion/Prediction\] explanation: \[Detailed Mechanism Analysis and Synthetic Proposal\]User Prompt:Task:Analyze the provided chemical reaction image\. Based on the specific reaction type and conditions shown, answer the following hypothetical reasoning question\.Question:\[\]Supplementary information:\[\]Question Type:Open\-Ended Reasoning / Reaction OptimizationSMILES OnlySupplementary Information \(Input\):Reactant1:COC\(=O\)c1ccc\(\-c2ccc\(Br\)cc2\)cc1,Reagents2:OB\(O\)c1ccccc1,Product1:COC\(=O\)c1ccc\(\-c2ccc\(\-c3ccccc3\)cc2\)cc1,Reagents:Pd\(OAc\)2,PPh3,TBAB;solvents:Toluene,H2OModel Output:answer: The reaction rate would decrease significantly, or the reaction might fail to proceed under the original mild conditions\. Chemical Reason: The C\-Cl bond is stronger than the C\-Br bond\. Consequently, the oxidative addition step—where the Pd\(0\) catalyst inserts into the carbon\-halogen bond—becomes much more difficult and acts as the rate\-determining step\.explanation: Theoretical Basis: The bond dissociation energy \(BDE\) of a C\-\-Cl bond \(∼\\sim339 kJ/mol\) is significantly higher than that of a C\-\-Br bond \(∼\\sim276 kJ/mol\), making the cleavage of the C\-\-Cl bond in oxidative addition less favorable thermodynamically and kinetically\. This increased bond strength results in a higher activation energy barrier for the oxidative addition step, which is the rate\-determining step in Suzuki\-Miyaura coupling reactions\.Mechanism Analysis: In the Suzuki coupling, the Pd\(0\) species undergoes oxidative addition with the aryl halide\. For bromides, this step proceeds readily due to the weaker C\-\-Br bond and better polarizability\. However, when the bromine is replaced by chlorine, the stronger C\-\-Cl bond resists oxidative addition, leading to slow or incomplete reaction\. Additionally, the lower nucleophilicity and poorer leaving\-group ability of chloride compared to bromide further hinder the process\.Synthetic Strategy: To overcome this challenge, electron\-rich and sterically bulky phosphine ligands are essential\. These ligands increase the electron density at the palladium center, enhancing its nucleophilicity and facilitating oxidative addition into the strong C\-\-Cl bond\. Specific examples include Buchwald\-type ligands such as SPhos, XPhos, or RuPhos, which are known for their high activity in aryl chlorides\. Alternatively, N\-heterocyclic carbene \(NHC\) ligands like IPr or SIPr can be used, as they provide strongσ\\sigma\-donation and stability to the Pd\(0\) complex\. The use of a more active catalyst system such as Pd\(OAc\)2with these advanced ligands, along with elevated temperatures or longer reaction times, may be required to achieve satisfactory yields\.ChemVA \(Ours\)Supplementary Information \(Input\):Reactant1:COC\(=O\)c1ccc\(\-c2ccc\(Br\)cc2\)cc1,Synonyms:methyl4’\-bromo\[1,1’\-biphenyl\]\-4\-carboxylate,IUPACName:methyl4\-\(4\-bromophenyl\)benzoate,Reagents2:OB\(O\)c1ccccc1,Synonyms:Phenylboronicacid,IUPACName:phenylboronicacid,Product1:COC\(=O\)c1ccc\(\-c2ccc\(\-c3ccccc3\)cc2\)cc1,Synonyms:,IUPACName:methyl4\-\(4\-phenylphenyl\)benzoate,Reagents:Pd\(OAc\)2,PPh3,TBAB;solvents:Toluene,H2OModel Output:answer: The reaction rate would decrease significantly, or the reaction might fail to proceed under the original mild conditions\. Chemical Reason: The C\-Cl bond is stronger than the C\-Br bond\. Consequently, the oxidative addition step—where the Pd\(0\) catalyst inserts into the carbon\-halogen bond—becomes much more difficult and acts as the rate\-determining step\.explanation: Theoretical Basis: The bond dissociation energy \(BDE\) of a C\-\-Cl bond \(∼\\sim339 kJ/mol\) is significantly higher than that of a C\-\-Br bond \(∼\\sim285 kJ/mol\), making oxidative addition of Pd\(0\) into a C\-\-Cl bond thermodynamically less favorable and kinetically slower\. This increased bond strength directly impacts the rate of the first step in the Suzuki\-Miyaura coupling mechanism, which is oxidative addition\. Additionally, chlorine is a poorer leaving group than bromine due to its lower polarizability and greater electronegativity, further reducing the rate of this step\.Mechanism Analysis: In the standard Suzuki coupling mechanism, oxidative addition is typically the rate\-determining step when using aryl halides\. With an aryl chloride instead of an aryl bromide, the electron\-deficient palladium center struggles to cleave the stronger C\-\-Cl bond\. The PPh3ligand used in the original conditions is moderately electron\-rich but not sufficiently so to stabilize the transition state for oxidative addition of a C\-\-Cl bond\. Moreover, PPh3has a moderate cone angle \(∼\\sim145∘\), which may not favor formation of the active monoligated Pd\(0\) species required for efficient oxidative addition\.Synthetic Strategy: To overcome this limitation, electron\-rich and sterically bulky phosphine ligands must be employed\. Buchwald\-type ligands such as SPhos, XPhos, or RuPhos are ideal because they increase electron density at the palladium center, facilitating oxidative addition of the strong C\-\-Cl bond\. These ligands also possess large cone angles \(\>\>150∘\), promoting the formation of low\-coordinate, highly reactive Pd\(0\) species\. Alternatively, N\-heterocyclic carbene \(NHC\) ligands like IPr or SIPr can be used due to their strongσ\\sigma\-donating ability, enhancing the nucleophilicity of Pd\(0\)\. A recommended optimized system would include Pd\(OAc\)2with XPhos \(or SPhos\) in a mixture of dioxane/water or toluene/water at elevated temperature \(80\-\-100∘C\) to achieve successful coupling\. Question 5 serves as a stress test for synthetic planning, asking the model to predict reactivity changes for an Aryl Chloride substrate and propose an optimization strategy\. - •Baseline: The baseline correctly identifies the primary thermodynamic barrier: the higher Bond Dissociation Energy \(BDE\) of C–Cl vs\. C–Br\. However, its reasoning isunidirectional, focusing solely on the substrate’s limitations\. It fails to explain specifically why the current ligand \(PPh3\) is insufficient beyond generic electronic arguments, offering only a standard textbook response\. - •ChemVA: It analyzes not just the substrate, but the specific incompatibility between the substrate and the ligand’s steric properties\. 1. \(1\)Mechanistic Depth:It explicitly cites the Cone Angle \(∼\\sim145∘\) of PPh3, reasoning that it is insufficient to promote the formation of the highly activemonoligatedPd\(0\) species required for difficult oxidative additions\. This incorporates advanced organometallic concepts \(steric bulk vs\. coordination number\) absent in the baseline\. 2. \(2\)Operational Precision:While the baseline suggests generic ligand classes, ChemVA proposes a complete experimental protocol, including specific solvents \(dioxane/water\) and temperature ranges \(80–100∘C\)\. This reflects a "tactical" understanding of reaction conditions that is crucial for wet\-lab success\. ### A4\.3\.The Role of Semantic Activation The qualitative superiority of ChemVA across Q3 \(Mechanism\), Q4 \(Context\), and Q5 \(Optimization\) highlights the critical function ofSemantic Activation\. Visual encodings \(or SMILES strings\) alone often trap Large Language Models in a "surface\-level retrieval" mode, where they generate generic facts based on pattern matching\. By translating visual features into explicit, knowledge\-rich textual descriptors \(e\.g\., IUPAC names, specific reagents like PPh3\), our method unlocks the model’s latent "expert subspace\." For instance, in Q5, the token "PPh3"—when explicitly presented via Semantic Activation—triggers associated concepts likeCone AnglesandTolman parametersfrom the model’s pre\-training corpus\. This allows the model to move beyond simple property retrieval \(BDE values\) to complex, multi\-variable reasoning \(steric\-electronic fits\), effectively bridging theSemantic Disconnectbetween visual perception and deep chemical reasoning\.
Similar Articles
Agentic generation of verifiable rules for deterministic, self-expanding reaction classification
This paper presents a multi-agent LLM pipeline that automatically generates and verifies reaction rules for chemical synthesis, expanding a standard taxonomy from 68 to 14,073 classes without human curation, achieving 97.7% classification accuracy on unseen reactions.
Back to Basics: Improving Molecular Understanding in LLMs via SMILES-Graph Translation
This paper proposes MolBasic, a structure-first framework that strengthens molecular understanding in LLMs via SMILES-Graph translation, using a multi-level structure perception benchmark and progressive learning with Chain-of-Thought to improve structural grounding and downstream task performance.
Large Language Model Teaches Visual Students: Cross-Modality Transfer of Fine-Grained Conceptual Knowledge
This paper introduces LaViD, a framework that transfers semantic knowledge from a language-only LLM to a vision student model by generating multiple-choice questions as conceptual signatures, achieving superior fine-grained classification performance and robustness.
LabVLA: Grounding Vision-Language-Action Models in Scientific Laboratories
LabVLA is a vision-language-action model for scientific laboratory automation, trained with a two-stage approach combining action token pretraining and flow matching. It achieves state-of-the-art success rates on the LabUtopia benchmark by leveraging simulated data to bridge the gap between household demonstrations and lab-specific tasks.
MechVQA: Benchmarking and Enhancing Multimodal LLMs on Comprehensive Mechanical Drawing Understanding
This paper introduces MechVQA, a dataset with 3.3k high-density mechanical engineering drawings and 21k question-answer pairs, along with the MechVL model that outperforms existing baselines by 7.57 percentage points on the MechVQA total score, advancing multimodal LLM understanding of mechanical drawings.