Multimodal Molecular Representation Learning with Graph Neural Networks, Deep & Cross Networks, and SMILES Embeddings

# Multimodal Molecular Representation Learning with Graph Neural Networks, Deep & Cross Networks, and SMILES Embeddings
Source: [https://arxiv.org/html/2607.05736](https://arxiv.org/html/2607.05736)
Chi Zhou2,∗\\astRuobing Wang1,2Zheng Ma1,21Department of Chemistry, Duke University, Durham, NC 27708, USA 2Department of Computer Science, Georgia Institute of Technology, Atlanta, GA 30332, USA ∗\\astThese authors contributed equally to this work\.

###### Abstract

Molecular property prediction often relies on isolated data modalities, where continuous 3D graph neural networks \(GNNs\) struggle to efficiently capture long\-range topological dependencies and exact macroscopic heuristics\. In this work, we introduce a parameter\-efficient Tri\-Branch Modular Fusion Neural Network that synthesizes three orthogonal modalities: 3D spatial geometry \(SchNet\), discrete topological grammar \(SMILES via ChemBERTa\), and explicit macroscopic physicochemical descriptors \(Deep & Cross Network\)\. By bypassing standard scalar readouts and employing a shared late\-fusion architecture, the framework establishes a mathematically rigorous multimodal latent space that effectively resolves the arithmetic and oversmoothing limitations of local message passing\. We evaluate the proposed architecture on the QM9 benchmark, targeting the extensive thermodynamic property of atomization energy at 0 K \(U0atomU\_\{0\}^\{\\mathrm\{atom\}\}\)\. Through systematic combinatorial ablation and latent bottleneck optimization \(de=64d\_\{e\}=64\), the tri\-modal framework achieves a validation Mean Absolute Error \(MAE\) of 0\.0207 eV\. Operating with fewer than one million parameters, this architecture decisively surpasses the sub\-chemical accuracy threshold and yields a substantial 20\.6% error reduction over a strictly controlled geometric baseline\. Ultimately, our findings demonstrate that integrating orthogonal macroscopic and topological data streams provides a synergistic,𝒪​\(1\)\\mathcal\{O\}\(1\)physical shortcut\. This multimodal alignment offers a highly efficient alternative to brute\-force parameter scaling, establishing a robust surrogate model for high\-throughput virtual screening \(HTVS\) pipelines\.

Keywords:Multimodal Fusion, Graph Neural Networks \(GNNs\), SchNet, Deep & Cross Networks \(DCNs\), Semantic Embeddings, Molecular Property Prediction, QM9\.

## 1Introduction

The accurate prediction of molecular properties is a fundamental task in computational chemistry, drug discovery, and materials science\. Properties such as atomization energy, dipole moments, and electronic characteristics are traditionally calculated using quantum mechanical methods like Density Functional Theory \(DFT\)\[[1](https://arxiv.org/html/2607.05736#bib.bib1)\]\. While DFT provides highly reliable predictions, its immense computational cost strictly limits its applicability in large\-scale workflows\. Consequently, machine learning \(ML\) has emerged as a critical surrogate, approximating DFT\-derived properties at a fraction of the computational expense\.

The development of molecular ML has been accelerated by standardized benchmarks like MoleculeNet\[[2](https://arxiv.org/html/2607.05736#bib.bib2)\]and generative models that map discrete representations to continuous latent spaces\[[3](https://arxiv.org/html/2607.05736#bib.bib3)\]\. Recently, geometric deep learning has dominated this space by learning directly from 3D spatial structures\. Foundational models like SchNet\[[4](https://arxiv.org/html/2607.05736#bib.bib4)\]demonstrated the power of continuous\-filter convolutions, while subsequent equivariant networks have achieved remarkable error floors on standard datasets like QM9\[[5](https://arxiv.org/html/2607.05736#bib.bib5)\]\.

However, this pursuit of absolute precision has driven the field toward heavily parameterized architectures\. Relying on complex high\-order tensors or massive attention mechanisms, state\-of\-the\-art equivariant models\[[6](https://arxiv.org/html/2607.05736#bib.bib6),[7](https://arxiv.org/html/2607.05736#bib.bib7)\]often require millions of parameters\. While these architectures achieve exceptional absolute accuracy, their computational overhead and high VRAM requirements present a bottleneck for real\-world applications like high\-throughput virtual screening \(HTVS\)\. In HTVS pipelines, where chemical libraries frequently exceed millions of candidate compounds, rapid inference and parameter efficiency are as critical as sub\-chemical accuracy\. This dynamic highlights a pressing need for alternative paradigms: achieving robust predictive fidelity not through the brute\-force parameter scaling of a single spatial modality, but through the computationally lightweight synthesis of complementary data sources\.

Monolithic, geometry\-only architectures often ignore the heterogeneous nature of chemical data\. Beyond 3D spatial coordinates, molecules are richly characterized by explicitly calculated physicochemical descriptors \(0D macroscopic scalars\) and structural sequences \(1D SMILES strings\)\[[8](https://arxiv.org/html/2607.05736#bib.bib8)\]\. While modern message\-passing GNNs excel at resolving localized quantum micro\-environments, they often struggle with exact global counting tasks and long\-range topological dependencies due to graph oversmoothing\. Conversely, tabular architectures \(like Deep & Cross Networks\) can easily map explicit macroscopic heuristics, and language models excel at capturing long\-range topological grammar via global self\-attention\. Combining these modalities theoretically provides an orthogonal mathematical shortcut that mitigates the blind spots of GNNs, though it remains a non\-trivial challenge due to stark differences in feature distributions and information density\[[10](https://arxiv.org/html/2607.05736#bib.bib10)\]\.

To address this, we propose a highly parameter\-efficient, multimodal representation learning framework that systematically integrates continuous geometric features, discrete semantic embeddings, and explicitly bounded physicochemical descriptors\. Our modular architecture processes each modality via a dedicated encoder before fusing them into a shared predictive latent space\. The geometric branch captures local interatomic dependencies using continuous\-filter convolutions adapted for extensive properties via additive pooling—a mathematical necessity for predicting extensive thermodynamic targets like atomization energy\. Concurrently, a Deep & Cross Network \(DCN\)\[[11](https://arxiv.org/html/2607.05736#bib.bib11)\]explicitly models global physicochemical heuristics via polynomial interactions\. Finally, the semantic branch distills pre\-trained SMILES embeddings via SwiGLU\-gated dimensionality reduction\[[12](https://arxiv.org/html/2607.05736#bib.bib12)\], capturing long\-range functional group motifs often obscured by localized Euclidean convolutions\.

Evaluated on the QM9 benchmark for the extensive atomization energy property, our main contributions are summarized as follows:

- •Parameter\-Efficient Multimodal Architecture:We propose a modular, tri\-branch fusion framework integrating a spatial GNN, explicit DCN feature crossing, and SwiGLU\-gated SMILES embeddings\. By relying on multimodal synergy rather than extreme depth, the framework operates with fewer than one million parameters, making it highly suitable for rapid inference pipelines\.
- •Sub\-Chemical Accuracy:The proposed fusion network achieves a validation Mean Absolute Error \(MAE\) of 0\.0207 eV on the QM9 benchmark, successfully surpassing the standard sub\-chemical accuracy threshold \(≈\\approx0\.0433 eV\) while maintaining a lightweight computational footprint\.
- •Synergistic Combinatorial Ablation:Through comprehensive ablation across unimodal and bimodal configurations, we empirically demonstrate that synthesizing orthogonal macroscopic and topological data yields a substantial 20\.6% reduction in error compared to a strictly controlled, pure geometric baseline \(0\.0261 eV\)\.
- •Latent Bottleneck Analysis:We provide a systematic sensitivity analysis of the semantic embedding dimensionality \(sweeping from 32 to 256\), identifying exactly 64 dimensions as the optimal information bottleneck required to distill global topology without over\-parameterizing the downstream latent space\.
- •Architectural Tabular Validation:We evaluate the tabular routing mechanism, confirming that explicit polynomial feature crossing via the DCN establishes a superior validation floor compared to implicit dense mixing via a standard MLP control \(0\.0210 eV\)\.

## 2Multimodal Learning Framework

### 2\.1Architecture Overview

The proposed Modular Fusion Network is designed to synthesize three orthogonal molecular modalities via a late\-fusion architecture\. This paradigm ensures each data stream is processed by a structurally optimized encoder, preventing feature interference during initial representation learning\. Formally, we define a moleculeMMas a multimodal tuple:

M=\(𝒢,𝒯,𝒮\)M=\(\\mathcal\{G\},\\mathcal\{T\},\\mathcal\{S\}\)\(1\)where𝒢=\(V,E\)\\mathcal\{G\}=\(V,E\)represents the 3D molecular graph,𝒯∈ℝdt​a​b\\mathcal\{T\}\\in\\mathbb\{R\}^\{d\_\{tab\}\}is a vector of macroscopic physicochemical descriptors, and𝒮∈ℝds​e​m\\mathcal\{S\}\\in\\mathbb\{R\}^\{d\_\{sem\}\}is the dense sequence embedding derived from a pretrained language model\.

As illustrated in Figure[1](https://arxiv.org/html/2607.05736#S2.F1), these modalities are routed through three independent parameterized encoders—Geometric \(Φg​e​o\\Phi\_\{geo\}\), Tabular \(Φt​a​b\\Phi\_\{tab\}\), and Semantic \(Φs​e​m\\Phi\_\{sem\}\)—to extract structurally aligned latent vectors of a uniform hidden dimensiondhd\_\{h\}\. This parallel routing funnels the orthogonal representations into a single unified state before projection to the target property\.

MoleculeMM3D Graph𝒢=\(V,E\)\\mathcal\{G\}=\(V,E\)Macroscopic𝒯∈ℝdt​a​b\\mathcal\{T\}\\in\\mathbb\{R\}^\{d\_\{tab\}\}SMILES𝒮∈ℝds​e​m\\mathcal\{S\}\\in\\mathbb\{R\}^\{d\_\{sem\}\}Geometric EncoderΦg​e​o\\Phi\_\{geo\}SchNet \+ Global PoolingTabular EncoderΦt​a​b\\Phi\_\{tab\}Deep & Cross NetworkSemantic EncoderΦs​e​m\\Phi\_\{sem\}SwiGLU \+ LayerNormConcat𝐳∈ℝ3​dh\\mathbf\{z\}\\in\\mathbb\{R\}^\{3d\_\{h\}\} Fusion MLPΨf​u​s​i​o​n\\Psi\_\{fusion\}Norm \+ LeakyReLUy^\\hat\{y\}\(Target\)𝐡g​e​o\\mathbf\{h\}\_\{geo\}𝐡t​a​b\\mathbf\{h\}\_\{tab\}𝐡s​e​m\\mathbf\{h\}\_\{sem\}

Figure 1:Schematic of the Tri\-Branch Modular Fusion Network\.The framework generalizes molecular representation by integrating a continuous\-filter GNN for 3D geometries, a DCN for explicit tabular physics, and a SwiGLU\-gated encoder for SMILES sequences\. The aligned latent representations \(𝐡g​e​o,𝐡t​a​b,𝐡s​e​m\\mathbf\{h\}\_\{geo\},\\mathbf\{h\}\_\{tab\},\\mathbf\{h\}\_\{sem\}\) are concatenated and processed through a shared late\-fusion network to predict the target extensive property \(y^\\hat\{y\}\)\.
### 2\.2Geometric Encoder \(Φg​e​o\\Phi\_\{geo\}\)

To capture the complex interatomic spatial dependencies required for quantum\-level predictions, our geometric branch utilizes a continuous\-filter convolutional architecture\[[4](https://arxiv.org/html/2607.05736#bib.bib4)\]\. Each atomi∈Vi\\in Vis associated with an atomic numberZiZ\_\{i\}and Cartesian coordinates𝐫i∈ℝ3\\mathbf\{r\}\_\{i\}\\in\\mathbb\{R\}^\{3\}\. Interatomic distancesdi​j=‖𝐫i−𝐫j‖2d\_\{ij\}=\\\|\\mathbf\{r\}\_\{i\}\-\\mathbf\{r\}\_\{j\}\\\|\_\{2\}parameterize a learnable continuous filter network to update node representations through iterative message passing\.

In this work, we select atomization energy at 0 K \(U0atomU\_\{0\}^\{\\mathrm\{atom\}\}\) as our primary predictive target to evaluate the framework\. To structurally adapt the architecture to this specific target, the geometric readout mechanism must be explicitly configured to match its underlying physics\. BecauseU0atomU\_\{0\}^\{\\mathrm\{atom\}\}is an extensive thermodynamic property—meaning its total magnitude scales directly with the number of atoms in the molecular system—standard size\-invariant readouts \(such as mean or max pooling\) would improperly collapse this critical size\-dependent information\.

Therefore, afterLLinteraction layers, we bypass standard readouts\. The raw spatial messages𝐡i\(L\)\\mathbf\{h\}\_\{i\}^\{\(L\)\}are first refined via an atom\-wise Multilayer Perceptron \(MLPatom\\mathrm\{MLP\}\_\{\\mathrm\{atom\}\}\) and then strictly aggregated using global additive pooling:

𝐡g​e​o=∑i=1\|V\|MLPatom​\(𝐡i\(L\)\)\\mathbf\{h\}\_\{geo\}=\\sum\_\{i=1\}^\{\|V\|\}\\mathrm\{MLP\}\_\{\\mathrm\{atom\}\}\\big\(\\mathbf\{h\}\_\{i\}^\{\(L\)\}\\big\)\(2\)By employing this additive constraint, the magnitude of the geometric latent representation \(𝐡g​e​o∈ℝdh\\mathbf\{h\}\_\{geo\}\\in\\mathbb\{R\}^\{d\_\{h\}\}\) scales proportionally with the atom count \(\|V\|\|V\|\), ensuring the resulting embedding space remains mathematically and physically consistent with extensive macroscopic properties\.

### 2\.3Tabular Encoder \(Φt​a​b\\Phi\_\{tab\}\)

To process the global physicochemical descriptors \(𝒯\\mathcal\{T\}\), we utilize a DCN\[[11](https://arxiv.org/html/2607.05736#bib.bib11)\]\. Unlike traditional MLPs that learn feature interactions implicitly, the DCN explicitly computes bounded polynomial cross\-feature dependencies, providing the network with exact macroscopic heuristics\.

For thell\-th cross layer, the interaction is formulated as:

𝐱l\+1=𝐱0⊙\(𝐖l⊤​𝐱l\)\+𝐛l\+𝐱l\\mathbf\{x\}\_\{l\+1\}=\\mathbf\{x\}\_\{0\}\\odot\(\\mathbf\{W\}\_\{l\}^\{\\top\}\\mathbf\{x\}\_\{l\}\)\+\\mathbf\{b\}\_\{l\}\+\\mathbf\{x\}\_\{l\}\(3\)where𝐱0\\mathbf\{x\}\_\{0\}is the standardized input vector, and𝐖l,𝐛l\\mathbf\{W\}\_\{l\},\\mathbf\{b\}\_\{l\}are learnable parameters\. Concurrently, a parallel deep network processes the input to extract implicit abstract patterns \(𝐱deep=MLPtab​\(𝐱0\)\\mathbf\{x\}\_\{\\text\{deep\}\}=\\mathrm\{MLP\}\_\{\\mathrm\{tab\}\}\(\\mathbf\{x\}\_\{0\}\)\)\. The outputs are concatenated and linearly projected:

𝐡t​a​b=𝐖final​\[𝐱Lc∥𝐱deep\]\+𝐛final\\mathbf\{h\}\_\{tab\}=\\mathbf\{W\}\_\{\\text\{final\}\}\[\\mathbf\{x\}\_\{L\_\{c\}\}\\parallel\\mathbf\{x\}\_\{\\text\{deep\}\}\]\+\\mathbf\{b\}\_\{\\text\{final\}\}\(4\)whereLcL\_\{c\}is the total number of cross layers, yielding the final tabular representation𝐡t​a​b∈ℝdh\\mathbf\{h\}\_\{tab\}\\in\\mathbb\{R\}^\{d\_\{h\}\}\.

### 2\.4Semantic Encoder \(Φs​e​m\\Phi\_\{sem\}\)

SMILES strings offer a robust topological mapping of molecular structure\. However, directly concatenating high\-dimensional textual embeddings introduces feature redundancy and risks drowning out the lower\-dimensional physical modalities\. To efficiently distill this embedding into the target latent space, we apply a SwiGLU\-gated projection mechanism\[[12](https://arxiv.org/html/2607.05736#bib.bib12)\]\.

The semantic vector is processed through parallel linear projections and gated via a Sigmoid Linear Unit \(SiLU\) activation, followed by dropout and layer normalization:

𝐡s​e​m=LayerNorm​\(Dropout​\(\(𝒮​𝐖1\)⊙SiLU​\(𝒮​𝐖2\)\)\)\\mathbf\{h\}\_\{sem\}=\\mathrm\{LayerNorm\}\\Big\(\\mathrm\{Dropout\}\\big\(\(\\mathcal\{S\}\\mathbf\{W\}\_\{1\}\)\\odot\\mathrm\{SiLU\}\(\\mathcal\{S\}\\mathbf\{W\}\_\{2\}\)\\big\)\\Big\)\(5\)where𝐖1,𝐖2∈ℝds​e​m×de\\mathbf\{W\}\_\{1\},\\mathbf\{W\}\_\{2\}\\in\\mathbb\{R\}^\{d\_\{sem\}\\times d\_\{e\}\}\. The intermediate semantic bottleneck dimension \(ded\_\{e\}\) acts as an informational strangulation point, systematically filtering linguistic noise while preserving critical long\-range topological dependencies before projecting todhd\_\{h\}\.

### 2\.5Multimodal Fusion and Optimization

The latent representations from the three distinct branches are concatenated to form a unified multimodal state \(𝐳∈ℝ3​dh\\mathbf\{z\}\\in\\mathbb\{R\}^\{3d\_\{h\}\}\):

𝐳=\[𝐡g​e​o​‖𝐡t​a​b‖​𝐡s​e​m\]\\mathbf\{z\}=\[\\mathbf\{h\}\_\{geo\}\\parallel\\mathbf\{h\}\_\{tab\}\\parallel\\mathbf\{h\}\_\{sem\}\]\(6\)This concatenated vector is processed through a deep fusion network \(Ψf​u​s​i​o​n\\Psi\_\{fusion\}\) utilizing Layer Normalization\[[17](https://arxiv.org/html/2607.05736#bib.bib17)\]and LeakyReLU activations to maintain gradient stability across the heterogeneous inputs, ultimately outputting the normalized predictiony^norm\\hat\{y\}\_\{\\mathrm\{norm\}\}\.

To ensure scale\-invariant gradient descent, target atomization energy values \(U0atomU\_\{0\}^\{\\mathrm\{atom\}\}\) are strictly standardized using training set statistics \(μy,σy\\mu\_\{y\},\\sigma\_\{y\}\)\. The framework is optimized end\-to\-end via the L1 Loss function \(Mean Absolute Error\):

ℒL1=1N​∑i=1N\|\(yi−μyσy\)−y^norm,i\|\\mathcal\{L\}\_\{\\mathrm\{L1\}\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\Big\|\\Big\(\\frac\{y\_\{i\}\-\\mu\_\{y\}\}\{\\sigma\_\{y\}\}\\Big\)\-\\hat\{y\}\_\{\\mathrm\{norm\},i\}\\Big\|\(7\)During inference, predictions are unnormalized \(y^=y^norm⋅σy\+μy\\hat\{y\}=\\hat\{y\}\_\{\\mathrm\{norm\}\}\\cdot\\sigma\_\{y\}\+\\mu\_\{y\}\) to compute the absolute error in physical units \(eV\)\.

## 3Experimental Setup

### 3\.1Dataset and Feature Engineering

To evaluate the predictive performance of our multimodal framework, experiments were conducted using the QM9 molecular benchmark\. To ensure strict data alignment across the spatial, physicochemical, and semantic domains, molecules that failed to generate valid 3D conformations, produced errors during RDKit descriptor calculation, or could not be tokenized by the semantic pipeline were systematically excluded\. This intersection protocol yielded a highly curated dataset of exactly 129,012 structurally aligned molecules, which was partitioned into training and validation subsets using a standardized 80%:20% split\.

For the tabular encoder \(Φt​a​b\\Phi\_\{tab\}\), a fixed vector ofdt​a​b=18d\_\{tab\}=18physicochemical properties is extracted for each molecule using the RDKit API\. As detailed in Table[1](https://arxiv.org/html/2607.05736#S3.T1), this feature set spans elemental composition, bond topology, global size metrics, and macroscopic heuristics\. All 18 descriptor features are standardized viazz\-score normalization calculated over the training partition to ensure stable gradient descent\.

Table 1:The 18 physicochemical and topological descriptors extracted via RDKit\.For the semantic encoder \(Φs​e​m\\Phi\_\{sem\}\), tokenized SMILES strings are processed throughChemBERTa\-77M\-MLM, a pretrained RoBERTa\-style foundation model\. This architecture fixes the input semantic dimensionality tods​e​m=384d\_\{sem\}=384, providing a contextualized sequence feature map for each molecule\.

Prediction Target and Normalization:In this work, the primary predictive target is the extensive atomization energy at 0 K \(U0atomU\_\{0\}^\{\\mathrm\{atom\}\}\)\. To ensure scale\-invariant optimization, ground\-truth targets arezz\-score normalized prior to training using global training set statistics \(μ=−76\.1823​eV,σ=10\.3195​eV\\mu=\-76\.1823\\text\{ eV\},\\sigma=10\.3195\\text\{ eV\}\)\. All final Mean Absolute Error \(MAE\) evaluations are unnormalized and reported in physical units \(eV\)\.

### 3\.2Implementation Details and Hyperparameters

To structurally align the feature spaces prior to late fusion, the hidden capacity for the geometric, tabular, and semantic pathways is fixed to a uniform dimension ofdh=128d\_\{h\}=128\. This constraint prevents the high\-dimensional semantic embeddings \(ds​e​m=384d\_\{sem\}=384\) from overpowering the multi\-source gradient flow\. Gradient stability across these heterogeneous modalities is further enforced via Layer Normalization \(LayerNorm\) at the bottlenecks of the tabular, semantic, and fusion networks\.

The geometric branch utilizesL=6L=6interaction blocks with 50 Gaussian filters and a cutoff radius of 10\.0 Å\. Crucially, to enable equitable late\-stage multimodal fusion, the standard SchNet scalar readout mechanism is bypassed\. Instead, the raw spatial messages are explicitly refined through a custom atom\-wise MLP \(128→64→128128\\rightarrow 64\\rightarrow 128\) with Softplus activations prior to global additive pooling\. While truncating the native readout slightly restricts the absolute peak performance of the standalone geometric model compared to highly tuned, monolithic packages, it is a deliberate and necessary architectural control\. It enforces strict latent parity \(dh=128d\_\{h\}=128\) across all modalities, ensuring that the fusion head measures the true synergistic gains of multimodal integration rather than being artificially dominated by an over\-parameterized spatial branch\.

Table 2:Detailed architectural hyperparameters and training configuration\.
### 3\.3Evaluation Metrics and Ablation Design

Generalization is evaluated on the validation partition using Mean Absolute Error \(MAE\) and the coefficient of determination \(R2R^\{2\}\)\. To guarantee a mathematically rigorous evaluation, all baseline and partial configurations are subjected to identical data splits, normalization parameters, and learning rate schedules as the proposed architecture\.

Rather than chasing absolute leaderboard dominance through parameter scaling, our primary objective is to investigate how explicitly distinct representations complement one another\. To systematically quantify this synergy, we establish an exhaustive, three\-phase ablation matrix:

Phase 1: Combinatorial Synergy\.Using the standalone 3D graph neural network \(SchNet Only\) as our controlled anchor baseline, we systematically activate, deactivate, and pair individual encoder pathways\. For this phase, the semantic projection bottleneck is strictly fixed tode=128d\_\{e\}=128to ensure perfect latent parity prior to fusion\. This isolates performance across seven configurations: all three unimodal baselines, all three dual\-modal pairings, and the full tri\-modal framework, directly measuring the orthogonal complementarity of the data sources\.

Phase 2: Semantic Bottleneck Sensitivity\.We hypothesize that directly projecting high\-dimensional text embeddings into a dense latent space risks over\-parameterizing the fusion head and introducing linguistic noise\. Freezing the internal geometric and tabular dimensions, we sweep the target compressed semantic dimension across a discrete grid \(de∈\{32,64,128,256\}d\_\{e\}\\in\\\{32,64,128,256\\\}\) to empirically identify the optimal information bottleneck required to distill global topology\.

Phase 3: Tabular Architectural Validation\.To verify that exact macroscopic heuristics require specialized mathematical routing, we isolate the tabular branch within the optimal tri\-modal framework\. We substitute the DCN cross\-network with a standard Multi\-Layer Perceptron \(MLP\) of equivalent parameter capacity\. This isolates whether explicit polynomial feature crossing establishes a superior error floor compared to implicit dense mixing\.

## 4Results and Discussion

### 4\.1Modality Ablation Analysis

To systematically verify the core hypothesis of this work—that geometric structures, tabular chemical descriptors, and text\-semantic sequences capture orthogonal, highly synergistic molecular features—we benchmark our multimodal configurations against the pure geometric baseline \(SchNet Only\)\. For this initial combinatorial phase, the semantic bottleneck is fixed tode=128d\_\{e\}=128to maintain strict latent alignment with the spatial and tabular branches\.

![Refer to caption](https://arxiv.org/html/2607.05736v1/x1.png)Figure 2:Multimodal Synergy Ablation\.Validation MAE convergence profiles demonstrating systematic error reduction via modality fusion\. Star markers \(⋆\\star\) indicate the optimally truncated checkpoint\. The tri\-modal convergence \(Panel c\) highlights both a significantly lower absolute error floor and accelerated, stabilized gradient dynamics compared to the isolated SchNet baseline\.Table 3:Combinatorial ablation performance evaluated at final convergence\. For these baseline comparisons, the semantic projection dimension is structurally aligned tode=128d\_\{e\}=128\.The experimental metrics, consolidated in Table[3](https://arxiv.org/html/2607.05736#S4.T3), establish a rigorous performance hierarchy that exposes the underlying information dynamics between the data sources\.

#### 4\.1\.1Unimodal Baselines and Contextual Constraints

Among isolated modalities \(Figure[2](https://arxiv.org/html/2607.05736#S4.F2)a\), spatial coordination data predictably provides the strongest unimodal signal\. TheSchNet Onlybaseline establishes a robust performance floor with a validation MAE of 0\.0261 eV\.

It is important to acknowledge that this absolute baseline error trails the highly optimized, standalone SchNet implementations frequently reported in state\-of\-the\-art literature\. This discrepancy is a deliberate byproduct of our strict experimental controls designed to measure multimodal synergy\. First, to enable equitable late\-stage fusion, the native scalar readout mechanism was truncated to output a high\-dimensional embedding \(dh=128d\_\{h\}=128\), sacrificing the deep, specialized atom\-wise energy networks typically employed in optimized packages \(e\.g\., SchNetPack\)\. Second, the training hyperparameters and learning rate schedules were globally standardized to ensure stable multi\-source gradient flow across the tri\-modal architecture, rather than being aggressively hyper\-tuned for the spatial branch in isolation\. Consequently, this baseline serves its intended purpose: acting as a strictly controlled anchor to isolate therelativesynergistic gains of multimodal integration, rather than competing with monolithic package\-level optimizations\.

When comparing the non\-geometric baselines, theEmbedding Onlymodel \(0\.2823 eV\) outperforms theDCN Onlymodel \(0\.3074 eV\)\. This gap is directly attributable to their pretraining paradigms: ChemBERTa captures structural syntax and functional group topology via masked language modeling, whereas the DCN relies on a highly compressed vector of predefined macroscopic scalars\. Unsurprisingly, without 3D spatial coordinates, these isolated modalities cannot resolve complex quantum energy landscapes, further validating the necessity of the geometric core\.

#### 4\.1\.2Dual\-Modal Synergies and Orthogonality

The true impact of multimodal representation emerges when evaluating the dual\-modal pairings\. Pairing modalities induces a striking, non\-linear error reduction driven by feature orthogonality \(Figure[2](https://arxiv.org/html/2607.05736#S4.F2)b\)\.

The most prominent evidence of this synergy is observed in theDCN \+ Embeddingconfiguration\. Despite completely lacking 3D atomic coordinates, fusing explicit physics \(DCN\) with global topology \(SMILES\) slashes the error nearly in half compared to their isolated states, achieving 0\.1458 eV\. This massive relative improvement strongly supports the hypothesis that macroscopic heuristics and topological sequences provide deeply complementary representations of the molecule\.

Even more revealing is the synergistic reversal that occurs when secondary modalities are fused with the 3D geometric core\. Although text outperformed tabular descriptors in isolation,SchNet \+ DCN\(0\.0222 eV\) yields a lower error thanSchNet \+ Embedding\(0\.0240 eV\)\. Theoretically, exact macroscopic properties \(e\.g\., atom counts, molecular weight\) are implicitly encoded within the 3D graph\. However, message\-passing GNNs are notoriously inefficient at exact global counting tasks due to oversmoothing\[[19](https://arxiv.org/html/2607.05736#bib.bib19),[21](https://arxiv.org/html/2607.05736#bib.bib21)\]\. By injecting explicitly calculated descriptors directly into the fusion head, the tabular branch acts as a physical shortcut\[[23](https://arxiv.org/html/2607.05736#bib.bib23)\]\. This pairing covers the GNN’s physical arithmetic blind spots, resulting in robust dual\-modal synergy\.

#### 4\.1\.3The Tri\-Modal Framework and Topological Scaling

When all three data streams are synthesized, the model reaches its highest predictive performance \(0\.0214 eV for the 128\-dimensional variant\)\. As visualized in Figure[2](https://arxiv.org/html/2607.05736#S4.F2)c, the tri\-modal architecture consistently outperforms all isolated and dual\-modality baselines, establishing a significantly lower absolute error floor\.

The mechanism driving this profound synergy lies in how the modalities structurally complement each other\. An important architectural question naturally arises: if SchNet inherently maps the 3D graph, why does integrating a 1D SMILES sequence yield further performance gains rather than causing redundant feature collapse?

The answer stems from the contrasting mathematical paradigms of their respective architectures\. SchNet resolves local quantum micro\-environments via message passing but struggles to efficiently exchange information between distant nodes\. Conversely, the semantic transformer processes topology globally; via self\-attention, the distance between any two tokens is effectively𝒪​\(1\)\\mathcal\{O\}\(1\)\[[26](https://arxiv.org/html/2607.05736#bib.bib26)\]\. Ultimately, the geometric branch maps localized micro\-physics, the tabular branch ensures exact macroscopic heuristics, and the semantic branch acts as a global topological grammar, seamlessly resolving the long\-range dependencies that pure 3D convolutions often obscure\.

### 4\.2Semantic Bottleneck Evaluation

While the tri\-modal architecture yields superior predictive capabilities, projecting 384\-dimensional text features directly into a 128\-dimensional latent space risks sub\-optimal feature mixing\. To maximize fusion synergy, we evaluated the SwiGLU projection layer by freezing the geometric and tabular latent spaces at 128 and sweeping the semantic bottleneck dimension \(ded\_\{e\}\) across a discrete search grid\.

Table 4:Impact of the semantic embedding bottleneck size \(ded\_\{e\}\) on final tri\-modal optimization\.Configuring the semantic bottleneck to exactlyde=64d\_\{e\}=64establishes the structural optimum, achieving the lowest validation MAE of 0\.0207 eV \(Table[4](https://arxiv.org/html/2607.05736#S4.T4)\)\. This represents asubstantial 20\.6% error reductioncompared to the unimodal geometric baseline\. The parabolic shape of the validation error confirms that synergistic fusion requires controlled feature compression\. Scaling past the optimal threshold \(de=256d\_\{e\}=256\) over\-parameterizes the projection, passing linguistic noise that drowns out the precise spatial coordinates\. Restricting the space to exactly 64 dimensions forces the network to distill only the most generalizable topological semantics\.

![Refer to caption](https://arxiv.org/html/2607.05736v1/x2.png)Figure 3:Framework Optimization\.\(a\)Validation trajectories across varying semantic bottleneck dimensions \(ded\_\{e\}\)\.\(b\)Architectural ablation of the tabular branch\. While explicit polynomial crossing \(DCN\) provides a marginal edge, the primary performance gain stems from the orthogonal macroscopic data itself\.#### 4\.2\.1Architectural Validation of the Tabular Branch

To rigorously evaluate whether the synergy is driven by thedataor thearchitecture, we conducted a structural control experiment\. Using the optimized tri\-modal framework \(de=64d\_\{e\}=64\), we substituted the explicit cross\-network \(DCN\) with a standard Multi\-Layer Perceptron \(MLP\) of equivalent parameter capacity\.

Table 5:Comparison of tabular routing architectures within the optimal framework \(de=64d\_\{e\}=64\)\.The MLP control achieved a highly competitive validation MAE of 0\.0210 eV, performing closely to the optimized DCN configuration \(0\.0207 eV\)\. This marginal performance delta confirms our foundational theory\. Standard MLPs are effective universal approximators for dense, low\-dimensional continuous spaces\[[25](https://arxiv.org/html/2607.05736#bib.bib25)\]\. Because our tabular branch relies on a compact vector of only 18 continuous descriptors, an MLP can implicitly approximate the necessary interactions\.

Consequently, while the DCN maintains a mathematical edge by explicitly bounding polynomial interactions, the MLP control proves the overarching thesis of this framework: the profound performance gains over standard spatial networks are driven primarily by the synergistic inclusion of orthogonal data modalities, rather than relying exclusively on complex architectural depth\.

### 4\.3Limitations and Architectural Trade\-offs

While the proposed tri\-modal architecture achieves a validation MAE of 0\.0207 eV for atomization energy \(U0atomU\_\{0\}^\{\\mathrm\{atom\}\}\)—comfortably surpassing the sub\-chemical accuracy threshold—several architectural constraints contextualize these findings and illuminate pathways for future refinement\.

First, to conduct a strictly controlled multimodal ablation, we deliberately constrained the geometric baseline\. The isolatedSchNet Onlyarchitecture was truncated via a custom atom\-wise MLP to force a 128\-dimensional latent bottleneck at the readout layer\. Although this modification successfully enforced structural parity—preventing the spatial branch from artificially dominating the multi\-source gradient flow—it restricted the standalone geometric model from reaching its theoretical peak performance\. Substituting this backbone with a massively parameterized equivariant network \(e\.g\., MACE or NequIP\) would undoubtedly depress the absolute error floor further\. However, such an upgrade would inherently sacrifice the sub\-million parameter footprint and rapid inference capabilities that define the core utility of this framework\.

Second, the semantic sequence branch utilizes a frozen, pretrained ChemBERTa language model\. While freezing these 77 million parameters critically prevents the high\-dimensional language model from overfitting and overpowering the lighter physical branches during early optimization, it relegates the transformer to a static feature extractor\. Consequently, the semantic encoder cannot fine\-tune its learned topological syntax specifically to theU0atomU\_\{0\}^\{\\mathrm\{atom\}\}thermodynamic energy landscape\. Future iterations could integrate Parameter\-Efficient Fine\-Tuning \(PEFT\) techniques, such as Low\-Rank Adaptation \(LoRA\), to enable end\-to\-end semantic fine\-tuning without violating the strict VRAM and parameter efficiency constraints of the architecture\.

Third, the empirical optimization dynamics reported in this study—including the identified semantic bottleneck \(de=64d\_\{e\}=64\) and the overall convergence trajectories—are inherently bounded by the defined hyperparameter search grid and available hardware compute\. While our specific configuration establishes a mathematically robust proof\-of\-concept for multimodal synergy, a more exhaustive, compute\-intensive Neural Architecture Search \(NAS\) across alternative fusion depths, learning rate schedules, and gating mechanisms could yield distinct optimization profiles\. Scaling this hyperparameter search utilizing distributed compute clusters remains a necessary future step to identify the absolute global minimum of the tri\-modal design space\.

Finally, the empirical evaluation is bounded by the inherent scope of the QM9 benchmark, which is restricted to small, equilibrium\-relaxed organic molecules \(up to nine heavy atoms\)\. As a result, the framework’s capacity to maintain synergistic multimodal alignment when scaling to large macromolecules, highly flexible poly\-pharmacological compounds, or non\-equilibrium transitional states remains an open question\. Future work will aim to extend this tri\-modal synthesis to dynamic molecular dynamics \(MD\) trajectories and structurally diverse datasets \(e\.g\., PCQM4Mv2\) to rigorously evaluate its predictive stability across shifting conformational landscapes\.

## 5Conclusion

In this work, we introduced a parameter\-efficient Tri\-Branch Modular Fusion Neural Network that systematically addresses the representational limitations of isolated molecular modalities\. By synthesizing continuous 3D spatial geometry \(SchNet\), discrete topological grammar \(SMILES via ChemBERTa\), and explicitly calculated macroscopic heuristics \(Deep & Cross Network\), we established a highly orthogonal, multimodal latent space for robust molecular property prediction\.

Empirical evaluation on the QM9 dataset demonstrates the exceptional predictive fidelity of this multimodal approach\. By optimizing the semantic projection bottleneck tode=64d\_\{e\}=64, the tri\-modal framework achieved a validation MAE of0\.02070\.0207eV for atomization energy \(U0atomU\_\{0\}^\{\\mathrm\{atom\}\}\)\. Operating with fewer than one million parameters, this architecture not only decisively surpasses the standard sub\-chemical accuracy threshold \(≈0\.0433\\approx 0\.0433eV\), but also achieves a substantial20\.6%20\.6\\%error reduction over a strictly controlled, pure geometric baseline\. This extreme parameter efficiency enables rapid inference, positioning the framework as a highly competitive surrogate for large\-scale, high\-throughput virtual screening \(HTVS\) pipelines\.

The success of this architecture is fundamentally grounded in information theory and physical complementarity\. While modern message\-passing graph neural networks excel at resolving continuous local micro\-environments, they inherently struggle with exact global counting tasks and often suffer from oversmoothing across long\-range dependencies\. Our ablation studies demonstrate that injecting tabular heuristics acts as a vital physical shortcut, effectively bypassing the network’s arithmetic bottlenecks\. Furthermore, integrating transformer\-based textual embeddings resolves macroscopic topology; by leveraging global self\-attention, it anchors the localized 3D graph with a codified,𝒪​\(1\)\\mathcal\{O\}\(1\)semantic prior\.

Looking forward, this multimodal alignment strategy provides a scalable blueprint for next\-generation molecular representation\. As the field gravitates toward massively over\-parameterized monolithic geometric models, our findings offer an alternative, highly efficient paradigm: fusing spatial networks with orthogonal macroscopic and topological data streams can effectively break current performance asymptotes without brute\-force parameter scaling\. Future work will explore parameter\-efficient fine\-tuning \(PEFT\) of the semantic encoder and aim to apply this tri\-modal framework to dynamic molecular trajectories, extending its utility from static equilibrium states to complex, non\-equilibrium thermodynamic environments\.

## Acknowledgements

The authors acknowledge the use of Google Gemini \(version Gemini 1\.5 Pro\) during the preparation of this manuscript\. Specifically, the AI was utilized as a writing assistant to polish the academic prose, ensure consistent mathematical formatting, and improve structural flow\.

## References

- \[1\]Kohn, W\., & Sham, L\. J\. \(1965\)\. Self\-consistent equations including exchange and correlation effects\.Physical Review, 140\(4A\)\.
- \[2\]Wu, Z\., et al\. \(2018\)\. MoleculeNet: a benchmark for molecular machine learning\.Chemical Science, 9\(2\), 513–530\.
- \[3\]Gómez\-Bombarelli, R\., et al\. \(2018\)\. Automatic chemical design using a data\-driven continuous representation of molecules\.ACS Central Science, 4\(2\), 268–276\.
- \[4\]Schütt, K\. T\., et al\. \(2017\)\. SchNet: A continuous\-filter convolutional neural network for modeling quantum interactions\.Advances in Neural Information Processing Systems, 30\.
- \[5\]Ramakrishnan, R\., et al\. \(2014\)\. Quantum chemistry structures and properties of 134 kilo molecules\.Scientific Data, 1, 1–19\.
- \[6\]Satorras, V\. G\., et al\. \(2021\)\. E\(n\) equivariant graph neural networks\.International Conference on Machine Learning, 9323–9332\.
- \[7\]Batatia, I\., et al\. \(2022\)\. MACE: Higher order equivariant message passing neural networks for fast and accurate force fields\.Advances in Neural Information Processing Systems, 35, 11423–11436\.
- \[8\]Weininger, D\. \(1988\)\. SMILES, a chemical language and information system\. 1\. Introduction to methodology and encoding rules\.Journal of Chemical Information and Computer Sciences, 28\(1\), 31–36\.
- \[9\]Wang, R\. \(2018\)\. Significantly improving the prediction of molecular atomization energies by an ensemble of machine learning algorithms and rescanning input space: A stacked generalization approach\.The Journal of Physical Chemistry C, 122\(9\), 4752–4760\.
- \[10\]Wang, Z\., et al\. \(2024\)\. Multi\-Modal Representation Learning for Molecular Property Prediction: Sequence, Graph, Geometry\.arXiv preprint arXiv:2401\.03369\.
- \[11\]Wang, R\., et al\. \(2017\)\. Deep & cross network for ad click predictions\.Proceedings of the ADKDD’17, 1–7\.
- \[12\]Shazeer, N\. \(2020\)\. GLU variants improve transformer\.arXiv preprint arXiv:2002\.05202\.
- \[13\]Kipf, T\. N\., & Welling, M\. \(2016\)\. Semi\-supervised classification with graph convolutional networks\.arXiv preprint arXiv:1609\.02907\.
- \[14\]Wang, R\., et al\. \(2021\)\. DCN V2: Improved Deep & Cross Network and Practical Lessons for Web\-scale Learning to Rank Systems\.Proceedings of the Web Conference\.
- \[15\]Chithrananda, S\., et al\. \(2020\)\. ChemBERTa: Large\-scale self\-supervised pretraining for molecular property prediction\.arXiv preprint arXiv:2010\.09885\.
- \[16\]Hendrycks, D\., & Gimpel, K\. \(2016\)\. Gaussian error linear units \(GELUs\)\.arXiv preprint arXiv:1606\.08415\.
- \[17\]Ba, J\. L\., et al\. \(2016\)\. Layer normalization\.arXiv preprint arXiv:1607\.06450\.
- \[18\]Maas, A\. L\., et al\. \(2013\)\. Rectifier nonlinearities improve neural network acoustic models\.Proceedings of ICML\.
- \[19\]Chen, Z\., et al\. \(2020\)\. Can graph neural networks count substructures?Advances in Neural Information Processing Systems\.
- \[20\]Wallace, E\., et al\. \(2019\)\. Do NLP models know numbers? Probing numeracy in embeddings\.Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)\.
- \[21\]Li, Q\., et al\. \(2018\)\. Deeper insights into graph convolutional networks for semi\-supervised learning\.Proceedings of the AAAI Conference on Artificial Intelligence\.
- \[22\]Oono, K\., & Suzuki, T\. \(2019\)\. Graph neural networks exponentially lose expressive power for node classification\.International Conference on Learning Representations \(ICLR\)\.
- \[23\]Karniadakis, G\. E\., et al\. \(2021\)\. Physics\-informed machine learning\.Nature Reviews Physics\.
- \[24\]Unke, O\. T\., et al\. \(2021\)\. Machine learning force fields\.Chemical Reviews\.
- \[25\]Gorishniy, Y\., Rubachev, I\., Khrulkov, V\., & Babenko, A\. \(2021\)\. Revisiting deep learning models for tabular data\.Advances in Neural Information Processing Systems, 34, 18932\-18943\.
- \[26\]Vaswani, A\., Shazeer, N\., Parmar, N\., Uszkoreit, J\., Jones, L\., Gomez, A\. N\., Kaiser, Ł\., & Polosukhin, I\. \(2017\)\. Attention is all you need\.Advances in Neural Information Processing Systems, 30, 5998\-6008\.