Generic Vision and Cross-Attention for Reaction Yield Prediction

# Generic Vision and Cross-Attention for Reaction Yield Prediction
Source: [https://arxiv.org/html/2608.00776](https://arxiv.org/html/2608.00776)
Qiwei HanDepartment of Chemistry, Duke University, Durham, NC 27708, USADepartment of Computer Science, Georgia Institute of Technology, Atlanta, GA 30332, USA

###### Abstract

Traditional reaction yield prediction is constrained by 1D quantum descriptors that lack explicit spatial information\. To address this gap, a dual\-modal Vision Cross\-Attention architecture is proposed, fusing tabular physical\-organic data with 2D molecular topologies\. Notably, it is demonstrated that a generic computer vision backbone processing simple 2D skeletal structures independently outperforms purely quantum\-based baselines\. By synergizing both modalities, superior predictive accuracy compared to traditional methodologies is achieved by the optimal cross\-attention framework \(Test RMSE = 5\.27%\)\. Through mechanistic probing, active, descriptor\-guided spatial querying is observed, effectively offloading macroscopic steric identification to the visual pathway\. Furthermore, a dynamic chemical hierarchy is learned by the network to heavily prioritize critical steric bottlenecks, such as the aryl halide\. Concurrently, residual skip connections are utilized to protect non\-spatial electronic parameters from destructive attenuation during fusion\. Collectively, a scalable and highly interpretable blueprint is provided for augmenting physical chemistry with deep visual learning\.

## 1Introduction

The integration of high\-throughput experimentation \(HTE\) with machine learning \(ML\) has fundamentally transformed the landscape of predictive synthetic chemistry\[[1](https://arxiv.org/html/2608.00776#bib.bib1),[20](https://arxiv.org/html/2608.00776#bib.bib2),[27](https://arxiv.org/html/2608.00776#bib.bib3),[5](https://arxiv.org/html/2608.00776#bib.bib4)\]\. The ability to accurately predict reaction yieldsin silicooffers a paradigm shift for reaction optimization, enabling researchers to virtually screen vast combinatorial chemical spaces while drastically reducing the time, cost, and material waste associated with empirical bench\-work\[[26](https://arxiv.org/html/2608.00776#bib.bib5),[25](https://arxiv.org/html/2608.00776#bib.bib6)\]\.

Historically, state\-of\-the\-art yield prediction models have relied heavily on explicit 1D molecular representations, such as Morgan fingerprints, reaction\-driven fingerprints \(DRFP\), or explicitly calculated quantum\-mechanical \(QM\) descriptors\[[21](https://arxiv.org/html/2608.00776#bib.bib7),[33](https://arxiv.org/html/2608.00776#bib.bib8),[23](https://arxiv.org/html/2608.00776#bib.bib9),[24](https://arxiv.org/html/2608.00776#bib.bib10)\]\. Seminal work by Doyle, Sigman, and others has demonstrated that concatenating molecular features—such as HOMO/LUMO energies, dipole moments, and Sterimol parameters—into tabular datasets can yield highly accurate predictive models when paired with algorithms like Random Forests or Multilayer Perceptrons \(MLPs\)\[[1](https://arxiv.org/html/2608.00776#bib.bib1),[8](https://arxiv.org/html/2608.00776#bib.bib11)\]\. However, this traditional tabular approach presents two significant bottlenecks\. First, the explicit calculation of high\-level Density Functional Theory \(DFT\) descriptors is computationally prohibitive for millions of theoretical screening candidates\[[10](https://arxiv.org/html/2608.00776#bib.bib12)\]\. Second, compressed 1D numerical arrays often struggle to holistically capture the complex, 2D and 3D steric environments and topological overlaps that govern transition\-state reactivity in a reaction mixture\[[17](https://arxiv.org/html/2608.00776#bib.bib13),[32](https://arxiv.org/html/2608.00776#bib.bib14)\]\.

To bypass the computational expense of QM descriptors, recent advances in chemical deep learning have increasingly turned to implicit spatial representations, most notably through computer vision\[[28](https://arxiv.org/html/2608.00776#bib.bib15),[22](https://arxiv.org/html/2608.00776#bib.bib16)\]\. Inspired by the success of Convolutional Neural Networks \(CNNs\) in image recognition, researchers have demonstrated that networks operating directly on 2D skeletal molecular drawings \(e\.g\., Chemception\) can successfully extract chemical heuristics—such as aromaticity, steric hindrance, and functional group presence—without requiring prior physical calculations\[[9](https://arxiv.org/html/2608.00776#bib.bib17),[18](https://arxiv.org/html/2608.00776#bib.bib18),[16](https://arxiv.org/html/2608.00776#bib.bib19)\]\. While these vision\-only approaches offer extraordinary computational scalability, they inherently lack the explicit quantum\-physical precision \(e\.g\., exact atomic charges\) provided by tabular data, often resulting in an informational ceiling when predicting highly sensitive catalytic yields\[[15](https://arxiv.org/html/2608.00776#bib.bib20),[4](https://arxiv.org/html/2608.00776#bib.bib21)\]\.

Consequently, the next frontier in chemical AI lies in multimodal architectures capable of bridging the gap between implicit spatial topologies \(vision\) and explicit physical parameters \(tabular data\)\[[11](https://arxiv.org/html/2608.00776#bib.bib22)\]\. To maximize this multimodal synergy, previous research has extensively explored cross\-attention mechanisms, demonstrating them to be a highly promising direction for fusing disparate data types\[[29](https://arxiv.org/html/2608.00776#bib.bib23),[14](https://arxiv.org/html/2608.00776#bib.bib24),[12](https://arxiv.org/html/2608.00776#bib.bib25)\]\. By allowing one modality to dynamically query the localized features of another, these attention\-based architectures have consistently achieved state\-of\-the\-art performance in complex multimodal tasks, making them the theoretical ideal for aligning 1D physical properties with 2D chemical structures\.

In this work, we introduce a dual\-modal architecture that synergizes explicit tabular QM data with high\-quality 2D molecular images to predict high\-throughput reaction yields\. Specifically, we sought to determine if an off\-the\-shelf, generic vision backbone \(ResNet\-18\)—completely devoid of domain\-specific chemical pre\-training—could extract sufficient topological information to rival explicitly calculated physics descriptors\. Furthermore, we conducted a rigorous architectural ablation study to investigate the underlying mechanisms of multimodal fusion, directly comparing dynamic cross\-attention against explicit vector concatenation \(Simple Concat\)\.

Notably, our results demonstrate that a generic vision model processing high\-quality 2D reaction topologies significantly outperforms a purely tabular MLP baseline anchored on computationally expensive quantum\-mechanical descriptors \(5\.60% vs\. 6\.79% RMSE\)\. By unifying these modalities through our optimal Vision Cross\-Attention framework, we successfully bridge 2D topology and 1D physical parameters to achieve superior predictive accuracy compared to traditional methodologies \(Test RMSE = 5\.27%\)\.

Beyond top\-level predictive accuracy, we conducted a mechanistic analysis to understand the internal logic of this multimodal approach\. Our findings indicate that rather than passively pooling visual data, the cross\-attention mechanism actively queries the 2D images based on the tabular descriptors, allowing it to efficiently map macroscopic steric environments\. Furthermore, analyzing the attention weights reveals that the network establishes a learned chemical hierarchy\. It exhibits highly targeted spatial routing, prioritizing critical steric bottlenecks \(e\.g\., the aryl halide\) while de\-emphasizing the topology of non\-critical components\. Finally, we observed a ”Residual Bottleneck,” highlighting that skip connections are mathematically necessary to preserve purely non\-spatial quantum parameters during the fusion process\. Overall, this framework provides a highly interpretable, computationally efficient blueprint for augmenting explicit physical chemistry with generic computer vision\.

## 2Computational Methods

![Refer to caption](https://arxiv.org/html/2608.00776v1/x1.png)Figure 1:The Dual\-Modal Vision Cross\-Attention Architecture\.2D spatial topologies extracted via a generic ResNet\-18 backbone are actively queried by 1D physical\-organic descriptors to dynamically contextualize the macroscopic reaction space\. A residual skip connection explicitly bypasses the spatial attention bottleneck to preserve strict quantum\-chemical rigor prior to the final yield prediction\.### 2\.1Dataset Partitioning and Modality Generation

We utilized the benchmarkrxnpredictdataset developed by the Doyle Lab\[[1](https://arxiv.org/html/2608.00776#bib.bib1)\], which maps high\-throughput reaction yields to rigorously calculated tabular physical\-organic descriptors \(e\.g\., HOMO/LUMO energies, dipole moments, and Sterimol parameters\)\. To ensure rigorous evaluation, the dataset was randomly partitioned into a 70:30 training and held\-out test split, utilizing a locked random seed to guarantee strict reproducibility across all architectural ablations\.

To facilitate the dual\-modal architecture, each reaction was parsed into two distinct representations:

- •Tabular Descriptors:The explicit 1D physical\-organic features were extracted and normalized using zero\-mean, unit\-variance standardization fitted exclusively on the training set to prevent data leakage\.
- •2D Visual Topologies:We generated standard 2D skeletal structure images for each individual reaction component—specifically the additive, aryl halide, base, and ligand—using RDKit\[[19](https://arxiv.org/html/2608.00776#bib.bib26)\]\. Images were standardized to a resolution of224×224224\\times 224pixels, converted to RGB tensors, and normalized using standard ImageNet parameters\. To ensure the network learned innate chemical topology rather than the arbitrary layout of a composite grid, these components were rendered and processed as independent images, aligned during training via a unique compound key \(experimental plate, row, and column\)\.

### 2\.2Modality Encoders

To process the multi\-modal inputs, we employed distinct neural network pathways for feature extraction prior to modality fusion\.

Tabular Pathway:The scaled physical\-organic descriptors were passed through a Feed\-Forward Multi\-Layer Perceptron \(MLP\) to generate a dense, dimensionally aligned tabular embedding of sized=256d=256\.

Vision Pathway:To extract 2D spatial features, we purposefully utilized a generic ResNet\-18 convolutional neural network\[[13](https://arxiv.org/html/2608.00776#bib.bib27)\]pre\-trained exclusively on ImageNet\. While domain\-specific models are tailored for physical chemistry, this backbone was designed for general\-purpose computer vision and was not specifically exposed to molecules during pre\-training\. We hypothesized that the network could implicitly learn 2D steric bulk and topological overlaps strictly from the geometric arrangement of the line\-angle drawings\. To isolate the cross\-attention learning dynamics, the convolutional backbone parameters were completely frozen\. The four reactant images were processed individually and stacked along the sequence dimension, yielding a discrete visual embedding matrixXv​i​s∈ℝ4×dkX\_\{vis\}\\in\\mathbb\{R\}^\{4\\times d\_\{k\}\}, allowing the downstream fusion mechanisms to interact with each specific reagent independently\.

### 2\.3Architectural Ablation and Fusion Strategies

To rigorously investigate the underlying mechanisms of feature fusion, we systematically evaluated two distinct mathematical approaches for bridging the explicit 1D tabular data with the implicit 2D visual embeddings: an explicit concatenation approach \(Simple Concat\) and a dynamic querying approach \(Cross\-Attention\)\.

Strategy A: Simple Concat Fusion \(The Baseline\) In the Simple Concat architecture, we bypass complex spatial routing in favor of a chemically blind, globally averaged topological state\. The component visual embeddings \(Xv​i​sX\_\{vis\}\) are collapsed via an unweighted Mean Pooling operation:

Xp​o​o​l=14​∑j=14Xv​i​s,jX\_\{pool\}=\\frac\{1\}\{4\}\\sum\_\{j=1\}^\{4\}X\_\{vis,j\}\(1\)This averaged visual state is then explicitly concatenated with the raw tabular descriptors \(Xt​a​bX\_\{tab\}\) and passed directly to the prediction head:

Xf​u​s​e​d=Concat​\(Xp​o​o​l,Xt​a​b\)X\_\{fused\}=\\text\{Concat\}\(X\_\{pool\},X\_\{tab\}\)\(2\)
Strategy B: Multi\-Head Cross\-Attention Fusion \(The Champion\) To test whether dynamic, descriptor\-guided spatial weighting could outperform a rigid global average, we designed a Multi\-Head Vision Cross\-Attention architecture \(Figure[1](https://arxiv.org/html/2608.00776#S2.F1)\)\. In standard attention paradigms, relevance scores are computed via the dot product of a Query \(QQ\) and a Key \(KK\), scaled and multiplied by a Value \(VV\)\[[29](https://arxiv.org/html/2608.00776#bib.bib23)\]\.

In our proposed architecture, the scaled physical\-organic tabular descriptors \(Xt​a​bX\_\{tab\}\) act as the central anchor \(the Query\), while the 2D visual embeddings extracted from the generic ResNet\-18 backbone \(Xv​i​sX\_\{vis\}\) act as the Keys and Values\. This explicitly allows the quantum math to actively search the visual topology\. To enable the network to simultaneously monitor distinct topological features, we execute spatial routing via Multi\-Head Cross\-Attention utilizingh=8h=8independent heads\. For each headi∈\{1,…,h\}i\\in\\\{1,\\dots,h\\\}, we compute the projections into a shared latent dimensiondk=256d\_\{k\}=256using learnable weight matricesWQ,iW\_\{Q,i\},WK,iW\_\{K,i\}, andWV,iW\_\{V,i\}:

Qi\\displaystyle Q\_\{i\}=Xt​a​b​WQ,i\\displaystyle=X\_\{tab\}W\_\{Q,i\}\(3\)Ki\\displaystyle K\_\{i\}=Xv​i​s​WK,i\\displaystyle=X\_\{vis\}W\_\{K,i\}\(4\)Vi\\displaystyle V\_\{i\}=Xv​i​s​WV,i\\displaystyle=X\_\{vis\}W\_\{V,i\}\(5\)
The cross\-attention for each independent head is subsequently computed by taking the scaled dot\-product of the explicit tabular query and the implicit visual keys, generating dynamic spatial routing weights:

headi=softmax​\(Qi​KiTdk\)​Vi\\text\{head\}\_\{i\}=\\text\{softmax\}\\left\(\\frac\{Q\_\{i\}K\_\{i\}^\{T\}\}\{\\sqrt\{d\_\{k\}\}\}\\right\)V\_\{i\}\(6\)
The outputs from allh=8h=8heads are then concatenated and projected through a final output weight matrixWOW^\{O\}to produce the unified vision\-contextualized embedding,Xa​t​t​nX\_\{attn\}:

Xa​t​t​n=Concat​\(head1,…,headh\)​WOX\_\{attn\}=\\text\{Concat\}\(\\text\{head\}\_\{1\},\\dots,\\text\{head\}\_\{h\}\)W^\{O\}\(7\)
Crucially, to prevent this spatial attention mechanism from acting as a destructive informational bottleneck for purely non\-spatial parameters \(e\.g\., electronic states or dipole moments\), we implemented a residual skip\-connection\[[13](https://arxiv.org/html/2608.00776#bib.bib27)\]\. The contextualized output from the multi\-head attention block \(Xa​t​t​nX\_\{attn\}\) is concatenated directly with the raw, unmodified tabular features \(Xt​a​bX\_\{tab\}\) to shield the rigorous quantum chemistry:

Xf​u​s​e​d=Concat​\(Xa​t​t​n,Xt​a​b\)X\_\{fused\}=\\text\{Concat\}\(X\_\{attn\},X\_\{tab\}\)\(8\)
Finally, for both architectural strategies, this fused representation is passed through a Multi\-Layer Perceptron \(MLP\) prediction head to output the continuous reaction yield prediction,y^\\hat\{y\}:

y^=MLP​\(Xf​u​s​e​d\)\\hat\{y\}=\\text\{MLP\}\(X\_\{fused\}\)\(9\)

### 2\.4Training Protocol and Interpretability Metrics

All networks were implemented in PyTorch and trained to predict continuous reaction yield \(%\) utilizing a Mean Squared Error \(MSE\) loss function\. Optimization was driven by the AdamW optimizer \(batch size = 32, initial learning rate =1×10−31\\times 10^\{\-3\}, weight decay =1×10−41\\times 10^\{\-4\}\), with the learning rate dynamically modulated via a Cosine Annealing scheduler with warm restarts\. Models were trained for 500 epochs, employing strict checkpointing to preserve the specific network weights that minimized held\-out validation loss\.

Top\-level predictive performance was quantified on the unseen test set using Root Mean Square Error \(RMSE\) and the coefficient of determination \(R2R^\{2\}\)\. Beyond predictive accuracy, we extracted specific interpretability metrics to determine mechanistic feature reliance\. Permutation Feature Importance \(PFI\) was calculated by individually shuffling each tabular feature array across the test set and recording the absolute resulting increase in RMSE\. Finally, for the spatial routing analysis, raw softmax attention scores were extracted directly from the cross\-attention layers during the forward pass over the test set, prior to multiplication with the Value \(VV\) matrix\.

## 3Results and Discussion

### 3\.1The Surprising Efficacy of Generic Vision and Modality Fusion

![Refer to caption](https://arxiv.org/html/2608.00776v1/x2.png)Figure 2:Evaluating Architectural Fusion and Modality Synergy\.Predictive performance \(RMSE andR2R^\{2\}\) across baseline and fused architectures\. The dual\-modal Cross\-Attention framework achieves the lowest error, seamlessly synergizing spatial visual topology with explicit tabular quantum descriptors\.We first sought to answer a provocative core hypothesis: can an off\-the\-shelf, generic vision model extract enough spatial topology from high\-quality 2D skeletal drawings to rival explicitly calculated quantum\-mechanical descriptors?

As shown in Figure[2](https://arxiv.org/html/2608.00776#S3.F2), the isolated Tabular MLP establishes a formidable baseline \(RMSE = 6\.80%\) by leveraging explicit 1D physical\-organic features \(e\.g\., HOMO/LUMO, dipole moments\)\. Notably, this neural network baseline alone already significantly outperforms the previous state\-of\-the\-art Random Forest benchmark established in the literature\[[1](https://arxiv.org/html/2608.00776#bib.bib1)\]\. However, when we evaluated the isolated Vision\-Only baseline—where high\-resolution 2D reaction topologies were processed through a frozen, ImageNet\-trained ResNet\-18 and aggregated via global mean pooling prior to final MLP prediction—the model achieved a significantly lower error \(RMSE = 5\.60%\)\.

This is a noteworthy observation\. It implies that in complex chemical reactions, macroscopic steric environments are complementary and equally vital to electronic states, though they often remain difficult to capture using purely 1D quantum arrays\. Conversely, the generic ResNet\-18 model appears adept at extracting geometric bulk directly from the images, despite lacking explicit chemical pre\-training\.

Interestingly, when we attempted a naive multimodal fusion \(the Simple Concat architecture\), the predictive performance was suboptimal, proving slightly worse than the Vision\-Only baseline\. This degradation highlights a well\-known pitfall in multimodal learning: modality dominance\[[30](https://arxiv.org/html/2608.00776#bib.bib28),[31](https://arxiv.org/html/2608.00776#bib.bib29)\]\. When passively pooled visual embeddings are directly concatenated with dense, explicit tabular parameters, the network naturally gravitates toward the easily interpretable tabular numbers during training\. Consequently, it dilutes the highly effective spatial representations learned by the vision backbone\.

To overcome this modality dominance, our Cross\-Attention framework eschews passive pooling in favor of an active, dynamic integration strategy\[[29](https://arxiv.org/html/2608.00776#bib.bib23),[14](https://arxiv.org/html/2608.00776#bib.bib24)\]\. Rather than treating the 1D quantum arrays and 2D visual embeddings as isolated vectors to be concatenated, the cross\-attention mechanism projects them into a shared latent space\. Specifically, the explicit tabular descriptors act as targeted mathematicalqueries, while the dense 2D spatial feature maps serve as thekeysandvalues\. This architectural bottleneck forces a strict inter\-modal dependency: the network cannot simply ignore the visual data during training because the 1D electronic features are explicitly tasked with retrieving complementary steric information from the 2D image pathway\. By structurally enforcing the equal importance of sterics and electronics without allowing one to overshadow the other, the framework successfully synergizes 1D physical properties with 2D spatial context, achieving a highly competitive predictive accuracy \(5\.27% RMSE\)\.

### 3\.2Mechanisms of Fusion: Shifting from Mass Proxies to Explicit Spatial Querying

![Refer to caption](https://arxiv.org/html/2608.00776v1/x3.png)Figure 3:Mechanistic shifts in feature reliance via Permutation Feature Importance \(PFI\)\.\(A\) Tabular Baseline:The MLP relies heavily onaryl halide molecular weightas a mathematical proxy for molecular size\.\(B\) Simple Concat:Passive mean pooling blurs spatial details, forcing the network to take a computational shortcut, falling back on molecular weight and purely electronic descriptors\.\(C\) Cross\-Attention:The mechanism executes a targeted spatial alignment\. By actively anchoring on explicit tabular shape parameters \(e\.g\., volume, ovality, surface area\), it successfully offloads steric identification to the 2D visual embeddings\.To elucidate the internal reasoning of the networks beyond top\-level performance metrics, we employed Permutation Feature Importance \(PFI\)\[[7](https://arxiv.org/html/2608.00776#bib.bib30)\]to map how the different architectures allocate predictive labor between the explicit quantum chemistry and the computer vision pathway \(Figure[3](https://arxiv.org/html/2608.00776#S3.F3)\)\.

In the isolated Tabular MLP \(Figure[3](https://arxiv.org/html/2608.00776#S3.F3)A\), the network exhibits a disproportionate reliance onaryl\_halide\_molecular\_weight\(\+9\.38% RMSE penalty\)\. Because the strictly 1D architecture lacks macroscopic spatial context, it is forced to utilize molecular weight as a general, mathematical heuristic to estimate steric bulk\.

Previously, we attributed the performance degradation of the Simple Concat architecture relative to the Vision\-Only baseline to modality dominance\. By analyzing its Permutation Feature Importance \(Figure[3](https://arxiv.org/html/2608.00776#S3.F3)B\), we uncover the precise computational behavior driving this dominance:shortcut learning\. Because the simple concatenated framework compresses the ResNet feature maps using unweighted mean pooling, critical localized spatial details are mathematically obscured\. Unlike the isolated Vision\-Only baseline, which is strictly forced to decipher visual features to minimize loss, the concatenated model is provided an alternative: the explicit tabular descriptors\. Consequently, the network takes a computational shortcut, preferentially extracting its predictive signal from the mathematically simpler tabular features rather than untangling the blurred visual embeddings\. This dynamic explains the shifting, yet persistent, importance of the aryl halide molecular weight\. Although the introduction of macroscopic visual data reduces its absolute dominance—dropping it from the top\-ranked feature to the fourth—molecular weight remains firmly within the top ten\. It survives as a necessary 1D heuristic proxy for steric bulk precisely because the network’s visual spatial resolution has been degraded, illustrating why simple concatenation struggles to fully synergize multimodal data\.

In stark contrast, the Cross\-Attention framework \(Figure[3](https://arxiv.org/html/2608.00776#S3.F3)C\) executes a highly active,descriptor\-guided spatial alignment\. Strikingly, molecular weight completely vanishes from its most critical features\. Instead, the tabular query relies strictly on explicit macroscopic shape descriptors—specificallymolecular\_volume,surface\_area, andovality\. The cross\-attention mechanism actively utilizes these physical parameters to query the generic ResNet topology, effectively tasking the visual pathway to localize the corresponding steric density within the 2D image\. This demonstrates that the cross\-attention architecture successfully and selectively offloads steric identification to the visual modality, rendering the fusion model exceptionally robust to tabular permutations\.

A critical question arises: why does the isolated Tabular MLP rely on molecular weight rather than the explicit geometric descriptors \(e\.g\., volume\) already present in the tabular dataset? This behavior stems from the interplay of feature variance and spatial blindness\. Molecular weight is an exact, invariant scalar, whereas computed molecular volume is intrinsically noisy, heavily dependent on the specific 3D conformer generation protocol\[[6](https://arxiv.org/html/2608.00776#bib.bib31),[2](https://arxiv.org/html/2608.00776#bib.bib32)\]\. Because a purely 1D architecture must treat its inputs as definitive answers—and lacks the spatial context to localize where an estimated volume physically resides—it preferentially anchors to the noiseless exactitude of molecular mass to approximate generalized bulk\.

However, within the Cross\-Attention framework, the role of tabular volume fundamentally shifts: it is no longer treated as a definitive scalar answer, but rather as a directional search query\. This aligns perfectly with the formal Query\-Key\-Value \(QKV\) routing established in modern attention mechanisms\[[29](https://arxiv.org/html/2608.00776#bib.bib23),[14](https://arxiv.org/html/2608.00776#bib.bib24),[3](https://arxiv.org/html/2608.00776#bib.bib33)\]\. Because geometric parameters like volume and surface area correlate directly with 2D topological area, the network utilizes these noisy tabular estimates as explicit mathematical queries to actively interrogate the visual pathway\. The high\-fidelity 2D image serves as the absolute structural verifier\. The model uses the rough volume estimate to search the visual keys, relying entirely on the image’s values to confirm, localize, and map the actual steric bulk\. By utilizing the visual modality to ground the noisy geometric query in absolute spatial reality, the cross\-attention mechanism successfully mitigates tabular conformer noise and renders the crude proxy of molecular weight obsolete\.

Finally, it is worth considering the behavior of the cross\-attention mechanism in data\-constrained scenarios where explicit geometric tabular descriptors \(e\.g\., volume and surface area\) are omitted from the dataset\. In such cases, neural networks exhibit opportunistic feature routing\. Deprived of direct geometric queries, the tabular encoder synthesizes a latent spatial query by aggregating secondary 1D proxies, such as molecular weight, heavy atom count, and implicit electronic signatures\. While this synthesized query is inherently less precise than explicit volume, the architectural integrity of the cross\-attention bottleneck remains intact\. The network still projects this proxy\-based query into the shared latent space to interrogate the visual pathway, allowing the 2D image to continuously function as the absolute structural verifier—compensating for the impoverished tabular query by mapping crude mass approximations directly onto the ground\-truth visual topology\.

### 3\.3Architectural Ablation and the Learned Chemical Hierarchy

![Refer to caption](https://arxiv.org/html/2608.00776v1/x4.png)Figure 4:Architectural ablation and visual verification of dynamic spatial specialization\.\(A\) The Residual Bottleneck:Forcing non\-spatial electronic descriptors entirely through the spatial attention block severely compromises predictive accuracy \(6\.17% RMSE\)\. Implementing a residual skip connection protects these quantum parameters from destructive attenuation, restoring superior predictive accuracy \(5\.27%\)\.\(B\) Raw Distribution by Head:The raw softmax attention scores from the 8 independent heads exhibit extreme variance \(0\.0 to 1\.0\), demonstrating sparse, binary\-like spatial routing rather than passive, uniform pooling\.\(C\) Learned Chemical Hierarchy:Tokenizing the visual input into four distinct reactant embeddings enables a direct mapping of attention scores to specific chemical components\. The network dynamically shifts its visual focus, heavily prioritizing the 2D topology of the Aryl Halide \(Mean = 0\.5327\) while almost entirely suppressing the Base \(Mean = 0\.0294\)\.To understand the specific design principles contributing to our champion 5\.27% RMSE, we performed a targeted architectural ablation, investigating both the information flow and the spatial attention distribution \(Figure[4](https://arxiv.org/html/2608.00776#S3.F4)\)\.

We first identified a critical phenomenon termed theResidual Bottleneck\(Figure[4](https://arxiv.org/html/2608.00776#S3.F4)A\)\. When explicit quantum descriptors are exclusively forced through the attention fusion block \(Cross\-Attention without Residual\), the model exhibits a severe accuracy penalty \(RMSE = 6\.17%\)\. This proves that while attention acts as an effective filter for querying physical shapes, it acts as a destructive informational bottleneck for purely non\-spatial quantum arrays \(e\.g\., dipole moments or HOMO energies\)\. The inclusion of a residual skip connection mathematically bypasses this bottleneck, shielding the unattenuated chemistry and culminating in the highly linear predictive correlation of the Champion architecture\.

Finally, to demystify how the network routes this spatial information, we systematically extracted the model’s internal attention mechanisms during the evaluation phase\. Specifically, we performed a complete forward pass over the unseen held\-out test set\. To ensure we captured the network’s pure spatial decision\-making process, we extracted the data using the standard scaled dot\-product attention formulation:Attention Weights = Softmax\(Q ×KTd\)By logging the raw attention tensors strictly at this computational step—immediately following the softmax normalization but prior to multiplication with the visual Value \(VV\) matrix—we isolate the unadulterated probability distributions generated by the 8 independent heads\. Unlike static model parameters, these extracted scores sum to 1\.0 and dictate exactly where the tabular query \(QQ\) mathematically ”looks” among the visual keys \(KK\) for any given reaction\.

The global distribution of these raw attention tensors across the test set \(Figure[4](https://arxiv.org/html/2608.00776#S3.F4)B\) reveals extreme variance\. Rather than defaulting to a uniform baseline, individual heads frequently output scores of absolute zero \(Median = 0\.0000\), indicating that the cross\-attention mechanism executes highly sparse, binary\-like spatial routing\.

Furthermore, because the visual pathway processes the reaction as four distinct image embeddings loaded in strict sequence \(Additive, Aryl Halide, Base, and Ligand\), we can directly map these logged attention probabilities back to specific chemical components\. By averaging the raw attention distributions across all 8 independent heads for every reaction in the test set, we calculated a singular global attention map for each prediction\. This aggregated data allowed us to quantify the network’s overall visual priority, establishing the learned chemical hierarchy shown in Figure[4](https://arxiv.org/html/2608.00776#S3.F4)C\.

Statistical analysis of these component\-mapped distributions refutes the hypothesis that the generic ResNet embeddings lack the resolution for targeted spatial querying\. The network entirely avoids passive mean pooling—which would yield a uniform 0\.25 attention score with zero variance across all components\. Instead, it establishes a strict, data\-driven chemical hierarchy\. The network heavily prioritizes the 2D topology of the Aryl Halide, allocating it 53\.27% of the global attention budget on average \(Mean = 0\.5327\)\. Strikingly, the Aryl Halide’s median attention score is 0\.5000, indicating that for at least half of all test reactions, the network dedicates the absolute majority of its visual focus exclusively to this single structural bottleneck\. Conversely, the 2D topology of the Base is completely suppressed \(Mean = 0\.0294\), with the network allocating it zero attention in over 75% of the dataset \(75th Percentile = 0\.0000\)\.

This highly polarized, sparse distribution proves that the cross\-attention mechanism is profoundly dynamic\. Guided by the explicit tabular quantum descriptors, the AI effectively learned which specific steric bottlenecks govern the reaction space, actively routing its generic visual pathway to the most critical structural components\.

## 4Limitations and Future Directions

While the dual\-modal Cross\-Attention architecture achieves highly competitive predictive performance, several limitations present clear avenues for future research\. First, although the generic ResNet\-18 excels at macroscopic spatial routing, it lacks explicit atomic precision\. Substituting this backbone with chemically pre\-trained vision foundation models could enable the microscopic localization of specific functional groups or transition\-state geometries\. Second, the framework relies on computationally expensive DFT\-derived quantum descriptors \(e\.g\., HOMO/LUMO energies\) to anchor its queries, limiting the high\-throughput screening of uncharted molecular scaffolds\. Future iterations could integrate Graph Neural Networks \(GNNs\) to rapidly approximate these quantum parameters, bypassing the DFT computational bottleneck\. Finally, while 2D topologies effectively capture general steric bulk, they inherently lack dynamic 3D conformational awareness\. Adapting this multimodal framework to incorporate computationally lightweight 3D point clouds or E\(3\)\-equivariant architectures remains a critical next step for modeling highly stereoselective reaction pathways\.

## 5Conclusion

In this study, we developed a dual\-modal Vision Cross\-Attention architecture that successfully bridges the gap between rigorous, explicitly calculated quantum\-chemical parameters and implicit 2D molecular topologies for high\-throughput reaction yield prediction\. By utilizing a parallel cross\-attention mechanism coupled with a protective residual skip connection, our framework effectively fuses 1D physical\-organic tabular descriptors with 2D spatial embeddings extracted via an off\-the\-shelf computer vision backbone \(ResNet\-18\)\.

Comprehensive evaluation on the benchmarkrxnpredictdataset yielded a compelling discovery: despite possessing strictly zero chemical pre\-training, a generic vision model processing high\-fidelity 2D geometric topologies significantly outperformed a standard baseline relying exclusively on computationally expensive quantum\-mechanical descriptors \(5\.60% vs\. 6\.79% Test RMSE\)\. Ultimately, unifying these representations in our optimal Cross\-Attention architecture achieved superior predictive accuracy compared to traditional methodologies \(RMSE = 5\.27%,R2R^\{2\}= 0\.961\), systematically resolving complex edge cases that isolated, single\-modality baselines struggled to contextualize\.

Beyond top\-level performance metrics, our mechanistic analyses illuminated the internal logic of multimodal chemical networks\. Through Permutation Feature Importance \(PFI\), we observed an elegant division of labor: the cross\-attention mechanism actively anchors on tabular shape descriptors \(e\.g\., volume and ovality\) to query the visual embeddings, effectively offloading macroscopic steric identification to the 2D pathway\. This frees the tabular network to focus exclusively on highly precise, non\-spatial electronic states\.

Furthermore, extracting the raw softmax attention scores debunked the assumption that generic vision models force chemical networks into passive pooling\. Instead, guided by explicit tabular queries, the network executes extreme, sparse spatial routing to establish a strict chemical hierarchy\. It dynamically dedicated the vast majority of its visual focus to critical steric bottlenecks \(e\.g\., the Aryl Halide\) while completely suppressing the visual topology of non\-critical components\. Finally, our architectural ablations demonstrated the absolute necessity of residual skip connections to bypass this spatial attention block, preventing the destructive attenuation of non\-spatial quantum parameters—a phenomenon we termed theResidual Bottleneck\.

Ultimately, this work challenges the prevailing assumption that highly specialized, domain\-specific pre\-training is an absolute prerequisite for chemical deep learning\. By demonstrating that generic visual networks can execute highly intelligent spatial routing when guided by explicit physical\-organic mathematics, we provide a synergistic, computationally lightweight blueprint for the ongoing development of fast, spatially aware predictive models in synthetic reaction optimization\.

## Acknowledgements

This research received no specific grant from any funding agency in the public, commercial, or not\-for\-profit sectors\. The authors gratefully acknowledge the Doyle Lab at Princeton University for generating and open\-sourcing therxnpredictdataset, which served as the foundational benchmark for this study\.

### Artificial Intelligence Disclosure

In accordance with academic publishing guidelines, the authors disclose the use of Google’s Gemini as an editorial assistant for text refinement and to assist in generating the schematic in Figure 1\. The authors take full responsibility for the final scientific content\. All quantitative data visualizations \(Figures 2, 3, and 4\) were generated programmatically via Python directly from raw experimental and model outputs\.

## References

- \[1\]D\. T\. Ahnemanet al\.\(2018\)Predicting reaction performance in c–n cross\-coupling using machine learning\.Science360,pp\. 186–190\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p1.1),[§1](https://arxiv.org/html/2608.00776#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.00776#S2.SS1.p1.1),[§3\.1](https://arxiv.org/html/2608.00776#S3.SS1.p2.1)\.
- \[2\]S\. Axelrodet al\.\(2022\)GEOM, energy\-annotated molecular conformations\.Sci\. Data9,pp\. 185\.Cited by:[§3\.2](https://arxiv.org/html/2608.00776#S3.SS2.p5.1)\.
- \[3\]N\. Carionet al\.\(2020\)End\-to\-end object detection with transformers\.InEur\. Conf\. Comput\. Vis\.,pp\. 213–229\.Cited by:[§3\.2](https://arxiv.org/html/2608.00776#S3.SS2.p6.1)\.
- \[4\]K\. V\. Chuanget al\.\(2018\)Comment on ”predicting reaction performance in c–n cross\-coupling using machine learning”\.Science362,pp\. eaat8603\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p3.1)\.
- \[5\]C\. W\. Coleyet al\.\(2020\)Autonomous discovery in the chemical sciences part ii: outlook\.Angew\. Chem\. Int\. Ed\.59,pp\. 23414–23436\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p1.1)\.
- \[6\]J\. Ebejeret al\.\(2012\)Freely available conformer generation methods: how good are they?\.J\. Chem\. Inf\. Model\.52,pp\. 1146–1158\.Cited by:[§3\.2](https://arxiv.org/html/2608.00776#S3.SS2.p5.1)\.
- \[7\]A\. Fisheret al\.\(2019\)All models are wrong, but many are useful: learning a variable’s importance\.J\. Mach\. Learn\. Res\.20,pp\. 1–81\.Cited by:[§3\.2](https://arxiv.org/html/2608.00776#S3.SS2.p1.1)\.
- \[8\]T\. Genschet al\.\(2022\)A comprehensive discovery platform for organophosphorus ligands for catalysis\.J\. Am\. Chem\. Soc\.144,pp\. 1205–1217\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p2.1)\.
- \[9\]G\. B\. Gohet al\.\(2017\)Chemception: a deep neural network with minimal chemistry knowledge\.arXiv preprint arXiv:1706\.06689\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p3.1)\.
- \[10\]C\. A\. Grambowet al\.\(2020\)Deep learning of activation energies\.J\. Phys\. Chem\. Lett\.11,pp\. 2992–2997\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p2.1)\.
- \[11\]J\. Guoet al\.\(2023\)Multimodal deep learning for chemical applications\.J\. Chem\. Inf\. Model\.63,pp\. 3561–3578\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p4.1)\.
- \[12\]Q\. Hanet al\.\(2026\)ChemFusion: a multimodal cross\-attention network for reaction yield prediction\.arXiv preprint arXiv:2607\.17033\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p4.1)\.
- \[13\]K\. Heet al\.\(2016\)Deep residual learning for image recognition\.InProc\. IEEE Conf\. Comput\. Vis\. Pattern Recognit\.,pp\. 770–778\.Cited by:[§2\.2](https://arxiv.org/html/2608.00776#S2.SS2.p3.1),[§2\.3](https://arxiv.org/html/2608.00776#S2.SS3.p8.2)\.
- \[14\]A\. Jaegleet al\.\(2021\)Perceiver: general perception with iterative attention\.InInt\. Conf\. Mach\. Learn\.,pp\. 4651–4664\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p4.1),[§3\.1](https://arxiv.org/html/2608.00776#S3.SS1.p5.1),[§3\.2](https://arxiv.org/html/2608.00776#S3.SS2.p6.1)\.
- \[15\]J\. P\. Janetet al\.\(2019\)A quantitative uncertainty metric controls error in neural network\-driven chemical discovery\.Chem\. Sci\.10,pp\. 7913–7922\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p3.1)\.
- \[16\]J\. Jiménez\-Lunaet al\.\(2020\)Drug discovery with explainable artificial intelligence\.Nat\. Mach\. Intell\.2,pp\. 573–584\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p3.1)\.
- \[17\]S\. Kearneset al\.\(2016\)Molecular graph convolutions: moving beyond fingerprints\.J\. Comput\.\-Aided Mol\. Des\.30,pp\. 595–608\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p2.1)\.
- \[18\]T\. B\. Kimberet al\.\(2021\)Deep learning in virtual screening: recent applications and developments\.Int\. J\. Mol\. Sci\.22,pp\. 4435\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p3.1)\.
- \[19\]G\. Landrumet al\.\(2024\)RDKit: open\-source cheminformatics software\.Cited by:[2nd item](https://arxiv.org/html/2608.00776#S2.I1.i2.p1.1)\.
- \[20\]D\. Pereraet al\.\(2018\)A platform for automated nanomole\-scale reaction screening and micromole\-scale synthesis in flow\.Science359,pp\. 429–434\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p1.1)\.
- \[21\]D\. Probstet al\.\(2022\)Reaction classification and yield prediction using the differential reaction fingerprint drfp\.Digit\. Discov\.1,pp\. 91–97\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p2.1)\.
- \[22\]K\. Rajanet al\.\(2021\)DECIMER: towards deep learning for chemical image recognition\.J\. Cheminf\.13,pp\. 1–9\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p3.1)\.
- \[23\]J\. P\. Reidet al\.\(2019\)Holistic predictive models of spatial and electronic requirements for ligand\-mediated selective catalysis\.Nature571,pp\. 343–348\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p2.1)\.
- \[24\]F\. Sandfortet al\.\(2020\)A structure\-based platform for predicting chemical reactivity\.Chem6,pp\. 1379–1390\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p2.1)\.
- \[25\]J\. Schwallkoet al\.\(2021\)Machine learning for chemical reactivity and reaction conditions\.Curr\. Opin\. Chem\. Biol\.65,pp\. 11–18\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p1.1)\.
- \[26\]B\. J\. Shieldset al\.\(2021\)Bayesian reaction optimization as a tool for chemical synthesis\.Nature590,pp\. 89–96\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p1.1)\.
- \[27\]F\. Strieth\-Kalthoffet al\.\(2020\)Machine learning the ropes: principles, applications and directions in synthetic chemistry\.Chem\. Soc\. Rev\.49,pp\. 6154–6168\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p1.1)\.
- \[28\]I\. V\. Tetkoet al\.\(2020\)BIGCHEM: challenges and opportunities for big data analysis in chemistry\.Mol\. Inform\.39,pp\. 1900132\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p3.1)\.
- \[29\]A\. Vaswaniet al\.\(2017\)Attention is all you need\.Adv\. Neural Inf\. Process\. Syst\.30\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p4.1),[§2\.3](https://arxiv.org/html/2608.00776#S2.SS3.p3.3),[§3\.1](https://arxiv.org/html/2608.00776#S3.SS1.p5.1),[§3\.2](https://arxiv.org/html/2608.00776#S3.SS2.p6.1)\.
- \[30\]W\. Wanget al\.\(2020\)What makes training multi\-modal classification networks hard?\.InProc\. IEEE/CVF Conf\. Comput\. Vis\. Pattern Recognit\.,pp\. 12695–12705\.Cited by:[§3\.1](https://arxiv.org/html/2608.00776#S3.SS1.p4.1)\.
- \[31\]N\. Wuet al\.\(2022\)Characterizing and overcoming the greedy nature of learning in multi\-modal representations\.InInt\. Conf\. Learn\. Represent\.,Cited by:[§3\.1](https://arxiv.org/html/2608.00776#S3.SS1.p4.1)\.
- \[32\]Z\. Wuet al\.\(2018\)MoleculeNet: a benchmark for molecular machine learning\.Chem\. Sci\.9,pp\. 513–530\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p2.1)\.
- \[33\]A\. F\. Zahrtet al\.\(2019\)Prediction of higher\-selectivity catalysts by computer\-driven workflow and machine learning\.Science363,pp\. eaau5631\.Cited by:[§1](https://arxiv.org/html/2608.00776#S1.p2.1)\.