Improving Molecular Property Prediction in Small Language Models Using Graph-based Tools

# Improving Molecular Property Prediction in Small Language Models Using Graph-based Tools
Source: [https://arxiv.org/html/2607.13115](https://arxiv.org/html/2607.13115)
\\copyrightclause

Copyright for this paper by its authors\. Use permitted under Creative Commons License Attribution 4\.0 International \(CC BY 4\.0\)\.

\\conference

2nd Causal Neuro\-symbolic Artificial Intelligence \(Causal NeSy\): Toward Agentic LLMs with Neuro\-Symbolic and Graph Based Reasoning

\[orcid=0000\-0002\-1910\-2758, email=bogas\.ko@iit\.demokritos\.gr, \]\\fnmark\[1\]\\cormark\[1\]

\[orcid=0000\-0002\-3434\-2717, \]\\fnmark\[1\]

\[\]\\cortext\[1\]Corresponding author\.\\fntext\[1\]These authors contributed equally\.

Konstantinos BougiatiotisInstitute of Informatics and Telecommunications, National Center for Scientific Research “Demokritos", Athens, GreeceGeorgios Paliouras

\(2022\)

###### Abstract

Small language models \(SLMs\) have shown promise for zero\-shot molecular property prediction fromSMILESstrings, yet they often suffer from*structural blindness*because sequence representations under\-specify key graph\-topological cues\. We propose a modularContext\-Augmented Promptingframework that enables agentic tool use at inference time: a trained GNN expert model provides a predictive hint with confidence, and aGNNExplainerextracts an instance\-specific explanatory subgraph \(e\.g\., a subgraphSMILESand an accompanying explanatory paragraph\)\. We evaluate three commonly used SLMs on MUTAG and Tox21 under five prompting configurations ranging from SMILES\-only to using all available tools at hand\. Across two datasets, enriching prompts with graph\-derived context yields substantial accuracy gains, often exceeding 25% relative improvement and up to 74% on Tox21\. We further validate the functional relevance of the extracted motifs via a necessity\-based edge\-drop intervention\. Despite the observed gains, a persistent gap remains to specialized GNN models, highlighting both the value and limits of text\-conditioned reasoning for molecular structure\.

###### keywords:

Molecule Property Prediction\\sepSmall Language Models \(SLMs\)\\sepGraph Neural Networks \(GNNs\)\\sepExplainability

## 1Introduction

Molecular Property Prediction \(MPP\) remains a foundational challenge in the drug discovery pipeline\[drug\_pipe,drug\_pipe1,drug\_pipe2\], where the accurate classification of chemical compounds determines the efficiency of the Design\-Make\-Test\-Analyze cycle\. Traditionally, Graph Neural Networks \(GNNs\) have emerged as the state\-of\-the\-art approach for these tasks due to their ability to directly model the topological structure of molecules\[gnn\_mols,gnn\_mols1,gnn\_mols2\]\. However, despite their predictive power, GNNs often operate as “black boxes," providing little human\-interpretable rationale for their outputs, which limits their utility in high\-stakes medicinal chemistry decisions\[xai\_survey,xai\_survey1\]\.

At the same time, Large Language Models \(LLMs\) have demonstrated a surprising capacity for zero\-shot reasoning across various scientific domains\. In the context of chemistry, LLMs typically process molecules via the Simplified Molecular Input Line Entry System \(SMILES\)\[smiles1\]\. While SMILES provides a compact string representation, it is a sequence\-based abstraction that frequently fails to encode complex 2D branching and spatial relationships\. Previous studies\[llm\_mols,llm\_mols1\]suggest that LLMs struggle with “structural blindness", leading to inconsistent performance when property prediction relies on specific local motifs or global connectivity\.

To address these limitations and showcase the improvement that LMs may exhibit when they have access to expert\-based tools, we propose aContext\-Augmented Promptingframework that leverages a GNN model to ground LM reasoning in the most influential subgraph identified by a GNN explainer\. By extracting the causal structural subgraph given by a GNN expert, and use them as a “hint” about chemical properties, we provide Small Language Models \(SLMs\) with the structural context that raw SMILES strings lack\.

In this work, we evaluate three prominent SLMs; namely Llama 3\.2 \(3B parameters\)\[llama\], Qwen 2\.5 \(3B parameters\)\[qwen2\.5\], and DeepSeek \(2\.4B active parameters in a 16B Mixture\-of\-Experts architecture\)\[deepseek\], across two benchmark datasets, MUTAG\[mutag\]and Tox21\[tox21\]\. Our empirical findings indicate that incorporating structured contextual information alongside baseline SMILES prompts can lead to meaningful performance improvements \(over25%25\\%relative improvement in almost all cases\)\. However, the effectiveness of such explanations appears to depend strongly on the underlying language model architecture, with smaller\-parameter models showing limited or inconsistent gains on certain tasks\. Additionally, although GNN\-augmented SLMs benefit from the integration of graph\-derived hints, they do not match the performance of specialized GNN architectures, highlighting persistent challenges in fully capturing structural information through text\-based reasoning alone\. The main contributions of this work are summarized as follows:

- •A Context\-Augmented Prompting Framework:We propose a strategy that bridges the gap between graph\-based structural learning and sequence\-based SLM reasoning by incorporating GNN\-derived hints\.
- •Quantification of Explanation Efficacy:We analyze the impact of the GNN\-derived signals on SLM performance, showing that while such context can substantially enhance predictive accuracy, the gains are dataset\- and model\-dependent, and a measurable gap remains between text\-augmented SLMs and specialized GNN architectures\.
- •Comparative SLM Benchmark:We provide an empirical evaluation of three state\-of\-the\-art SLMs on molecular classification, highlighting the varying degrees of “structural blindness" inherent in different model architectures\.

## 2Related Work

### 2\.1Structural Representation Learning in Chemistry

The evolution of Molecular Property Prediction \(MPP\) has transitioned from handcrafted molecular descriptors and fingerprints, such as ECFP\[Rogers2010\], to deep representation learning\. Graph Neural Networks \(GNNs\), including Graph Convolutional Networks \(GCN\)\[gcn\]and Graph Isomorphism Networks \(GIN\)\[Xu2019\], are widely adopted for molecular representation learning, by treating molecules as topological graphs\. While these models excel at capturing local and global chemical environments, their internal representations remain non\-symbolic, making it difficult for practitioners to extract actionable chemical insights\.

### 2\.2Explainability in Graph Neural Networks

To address the opacity of GNNs, various post\-hoc explanation methods have been proposed\. Tools such as GNNExplainer\[Ying2019\]and PGExplainer\[Luo2020\]identify the most influential subgraphs \(motifs\) contributing to a specific prediction\. However, these explanations are typically provided as importance weights over edges or nodes\. Transforming these numerical importance scores into human\-intelligible reasoning remains a challenge, which this work addresses by using SLMs as a linguistic interface\.

### 2\.3LLMs for Molecular Reasoning

Recent advancements have seen LLMs applied to chemistry via SMILES string processing\. Models like Galactica\[Taylor2022\]and MolT5\[Edwards2022\]demonstrated that pre\-training on large scientific corpora enables basic property prediction\. However, research into general\-purpose models like Llama 3 and GPT\-4 has revealed a “SMILES\-to\-structure" gap; where these models often fail to reliably reconstruct or reason over the underlying molecular graph topology\[chembert\]\. Our work builds on this by testing if explicit structural “hints" can mitigate this sequential bias\.

### 2\.4Hybrid GNN\-SLM Integration

Emerging research has explored the alignment of graph encoders with language models, often through contrastive learning or projection layers, such works include MolCA\[Molca\]or MolLM\[MolLM\]\. Unlike these computationally expensive architectural integrations, our approach focuses onContext\-Augmented Prompting\. This method preserves the frozen weights of state\-of\-the\-art SLMs and investigates the extent to which GNN\-derived explanatory signals, representing the model\-identified most influential substructures, can help mitigate the modality gap\. Note that this strategy is both more accessible and modular for practical deployment\.

## 3Methodology

### 3\.1Problem Setting

We study binary molecular property prediction from a textual representation\. Each molecule is provided as aSMILESstringss, and the target label isy∈\{0,1\}y\\in\\\{0,1\\\}indicating whether the molecule is toxic/mutagenic under the dataset definition\. Our goal is to improve*zero\-shot*performance of SLMs on this task by enabling*agentic tool use*: at inference time, the SLM can query a learned graph expert and explanation tool to obtain structured context that is otherwise absent from theSMILESsequence\.

### 3\.2Context\-Augmented Prompting Framework

As shown in Figure[1](https://arxiv.org/html/2607.13115#S3.F1), our framework adopts an agentic architecture where an SLM serves as the reasoning controller, utilizing tools to interact with a GNN expert\.

![Refer to caption](https://arxiv.org/html/2607.13115v1/images/workflow.png)Figure 1:The proposed agentic loop\. Given the query SMILES, the SLM invokes one or more of the available tools to interact with pre\-trained GNN expert, to generate related context and augment the original prompt, to provide assistance for the final inference\.First, let a molecule represented by theSMILESsscorrespond to a molecular graphG=\(V,E,𝐗\)G=\(V,E,\\mathbf\{X\}\)with atomsVV, bondsEE, and node features𝐗\\mathbf\{X\}\(e\.g\., atom type, valence, aromaticity\)\. We assume that there is pre\-trained GNN \(expert\) model available to us111Practically, we train aGINmodelfθ​\(G\)→\(y^,p^\)f\_\{\\theta\}\(G\)\\rightarrow\(\\hat\{y\},\\hat\{p\}\)on the training split of each dataset, withGGa molecule graph,y^∈\{0,1\}\\hat\{y\}\\in\\\{0,1\\\}andp^∈\[0,1\]\\hat\{p\}\\in\[0,1\]is the model confidence\.\. We model inference as a single\-step agent loop: given an inputSMILESstringss, the SLM can call tools that return \(1\) expert predictions and confidence \(y^,p^\\hat\{y\},\\hat\{p\}\), \(2\) the determined explanatory subgraph \(GcG\_\{c\}\), and \(3\) a natural\-language rationale derived from the subgraph \(rr\)\. Specifically, we define three primary tools that the SLM can invoke to enhance its zero\-shot reasoning:

- •ExpertPredict\(s\)\(s\): convertss↦Gs\\mapsto G, returns\(y^,p^\)\(\\hat\{y\},\\hat\{p\}\)from the pre\-trainedGIN\.
- •ExplainSubgraph\(s\)\(s\): returns an explanatory subgraphGcG\_\{c\}\(in the form of the subgraphSMILESscs\_\{c\}\) usingGNNExplainer\[Ying2019\]over the sameGIN\.
- •ChemReason\(s,sc,y^,p^\)\(s,s\_\{c\},\\hat\{y\},\\hat\{p\}\): using a short chemistry\-oriented explanation templated prompt, we instruct the SLM to reason about why the motifscs\_\{c\}might support toxicity/mutagenicity\. In practice we implement this as an instruction block and ask the SLM itself to generate the rationale; conceptually it is a tool interface for a reasoning module\.

To avoid introducing external knowledge bias from larger teacher models, each SLM model itself is used to generate the rationale \(i\.e\., ChemReason\), ensuring that the performance gains are strictly a product of the model’s own reasoning conditioned on GNN\-derived evidence\.

RegardingExplainSubgraph, we extract a per\-instance explanatory subgraph viaGNNExplaineras an edge\-importance mask𝐦∈\[0,1\]\|E\|\\mathbf\{m\}\\in\[0,1\]^\{\|E\|\}\. We then define the*candidate causal subgraph*:

Gc=\(Vc,Ec\),Ec=\{e∈E:me≥τ\},G\_\{c\}=\(V\_\{c\},E\_\{c\}\),\\quad E\_\{c\}=\\\{e\\in E:m\_\{e\}\\geq\\tau\\\},where the thresholdτ\\tauis chosen dynamically for each module, so thatEcE\_\{c\}contains only edges at the top10%10\\%of the edge importance score\. That is, the designated subgraphGcG\_\{c\}contains only the top\-10%10\\%most important edges of the graph, as identified by GNNExplainer\. To makeGcG\_\{c\}consumable by text\-only SLMs, we convert it to aSMILESstring \(denotedscs\_\{c\}\)\.

### 3\.3Prompt Configurations

Using the above tools, we evaluate five prompt configurations that correspond to progressively richer tool outputs\. The variants used are shown below \(the exact text of these different prompt configurations can be seen in Appendix[A](https://arxiv.org/html/2607.13115#A1)\):

1. 1\.SMILES \(Baseline\)\.Input is only theSMILESstringss\.
2. 2\.SMILES \+ SUBGRAPH\.Input includesssand the extracted subgraphSMILESscs\_\{c\}\.
3. 3\.SMILES \+ HINT\.Input includesssand the GNN expert output\(y^,p^\)\(\\hat\{y\},\\hat\{p\}\)as a “field expert hint” \(i\.e\., the GNN label prediction and confidence in%\\%\) \.
4. 4\.SMILES \+ REASONING\.Input includesssand a short natural\-language rationale about the motif’s relevancerr\(generated by the SLM when givenssandscs\_\{c\}\)\.
5. 5\.ALL CONTEXT\.Input includesss,\(y^,p^\)\(\\hat\{y\},\\hat\{p\}\),scs\_\{c\}and the rationalerr\.

## 4Experiments and Discussion

### 4\.1Experimental Setup

We evaluate our framework on two standard molecular classification benchmarks: MUTAG\[mutag\]and a random subset of 1,000 samples from Tox21\[tox21\]\.MUTAGcontains 188 nitroaromatic and heteroaromatic compounds labeled for mutagenicity, serving as a small benchmark for graph\-based molecular classification\.Tox21is a multi\-label toxicity benchmark that spans multiple biological targets; we sample 1,000 molecules and convert the task to binary classification by marking a molecule as positive if it has*any*positive toxicity label\. For each dataset, we use a 70/30 train/test split\. Importantly, the split is used*only*to train theGINexpert\. The SLMs are evaluated in a*zero\-shot*manner on the test portion only\. The general characteristics of the datasets are shown in Table[1](https://arxiv.org/html/2607.13115#S4.T1)\.

Table 1:Dataset characteristics\.Dataset\# Molecules/GraphsAvg\. NodesAvg\. EdgesPositive Ratio \(%\)MUTAG18818\.0339\.8067Tox21100018\.3738\.9937For the GNN expert, we adopt a commonGINsetup across datasets to reduce confounding: 3 message\-passing layers, hidden dimension 64, a learning rate of2⋅10−32\\cdot 10^\{\-3\}and trained for up to 100 epochs with early stopping\. We hold out 10% of the training split for validation and use patience 10 for early stopping\.

### 4\.2Small Language Models

We evaluate three SLMs spanning different architectural and capacity regimes:DeepSeek\(Lite\-Code\), a Mixture\-of\-Experts model with 16B total parameters and 2\.4B active parameters;Qwen 2\.5\(Code\), with 3B parameters; andLlama 3\.2, also with 3B parameters\.DeepSeekis optimized for code\-centric instruction following and structured reasoning\.Qwen 2\.5is likewise tuned for step\-wise generation and robust formatting, making it well suited to evidence\-conditioned decision prompts\.Llama 3\.2serves as a general\-purpose baseline, enabling comparison between code\-tuned models and a more broadly trained conversational model under identical zero\-shot prompting\. All models are evaluated in zero\-shot mode on the test splits of the datasets, using fixed prompts for each configuration\. We report classificationaccuracy\(%\) for each prompt configuration\.

### 4\.3Main Results

Table[2](https://arxiv.org/html/2607.13115#S4.T2)summarizes the accuracy over the different prompt configurations and the two datasets\.

Table 2:Model accuracy \(in %\) across different prompt configurations\. The highest accuracy per SLM and dataset is highlighted in bold\. The “Improvement” row indicates the relative percentage gain of the ALL CONTEXT configuration compared to the baseline SMILES prompt\.ConfigurationMUTAGTox21DeepSeekQwen 2\.5Llama 3\.2DeepSeekQwen 2\.5Llama 3\.2SMILES56\.1455\.3647\.3738\.5047\.5048\.50SMILES \+ SUBGRAPH54\.3966\.0743\.8645\.5047\.2455\.00SMILES \+ HINT42\.1149\.1243\.8667\.0067\.5064\.00SMILES \+ REASONING66\.6749\.1240\.3555\.5051\.5061\.50ALL CONTEXT75\.4470\.1847\.3767\.0068\.5063\.50Improvement over SMILES↑\\uparrow34\.38%↑\\uparrow26\.77%0\.00↑\\uparrow74\.03%↑\\uparrow44\.21%↑\\uparrow30\.93%GNN Expert84\.2171\.00#### Observation 1: Tool augmentation yields large gains on the harder setting\.

OnTox21, the baseline SMILES\-only prompting is low for DeepSeek \(38\.50%\) and moderate for Qwen/Llama \(47–49%\)\. Adding tools closes much of this gap, with ALL CONTEXT reaching 67\.00% \(DeepSeek\) and 68\.50% \(Qwen\), representing large relative gains \(74\.03% and 44\.21% relative improvements over baseline, respectively\)\. This is consistent with the claim that rawSMILESinduces “structural blindness” while graph\-derived context restores missing relational information\.

#### Observation 2: Model capacity/competence mediates benefit from structured context\.

The strongest and most consistent improvements are observed for DeepSeek and Qwen\. Llama 3\.2 shows mixed behavior: some single\-tool variants help onTox21\(e\.g\., SUBGRAPH and HINT\), but the ALL CONTEXT improvement is smaller and inMUTAGdoes not improve beyond baseline, suggesting that smaller\-scale general\-purpose models may suffer from “information distraction"\. This suggests that benefiting from tool outputs requires the SLM to \(i\) integrate heterogeneous evidence and \(ii\) maintain instruction\-following under longer contexts\.

#### Observation 3: Single\-tool signals can be unstable; combined context is more robust\.

Across datasets, isolated signals sometimes degrade performance \(e\.g\., MUTAG with SMILES\+HINT for DeepSeek\)\. This is expected in an agentic setting: expert hints may be miscalibrated, and subgraphs may be hard to interpret without narrative scaffolding\. ALL CONTEXT often recovers and improves, indicating complementary effects:*\(a\) expert prediction provides a prior*,*\(b\) motif provides structural evidence*,*\(c\) reasoning promotes semantic alignment*between text and graph evidence\.

#### Observation 4: The gap to specialized graph models remains\.

Even with ALL CONTEXT, SLMs do not consistently surpass theGINexpert \(84\.21% on MUTAG; 71\.00% on Tox21\)\. This highlights a boundary of purely text\-conditioned reasoning: graph inductive bias and representation learning remain crucial for molecular structure\.

### 4\.4Complementary Experiments and a Qualitative Case Study

![Refer to caption](https://arxiv.org/html/2607.13115v1/images/necessity_plot_new.png)Figure 2:Necessity test, showing theGINaccuracy as edges are removed in explainer\-importance order \(red\) vs random removal \(grey\)\. A larger separation indicates that the extracted motif is closer to a necessary substructure for the expert prediction\.In this section, we first present a supplementary experiment examining the necessity of the highlighted subgraphs identified by GNNExplainer, thereby supporting the importance of the selection\. We then illustrate, through a sample\-based case study, a scenario in which the SLM, using the reasoning tool, can disregard the incorrect input provided by the GNNExpert\.

To assess whether the extracted motif is merely correlational or behaves like a*necessary*substructure for the expert’s prediction, we perform a targeted perturbation test on the expert model: we progressively remove edges fromGGeither: \(i\) in decreasing importance order according toGNNExplainer\(GNNExplainer\), or \(ii\) at random \(Random Score\), and measure the expert accuracy as a function of the number of removed edges\. A strong separation between these curves indicates that explainer\-ranked edges concentrate predictive signal and removing them is a stronger intervention, consistent with a necessity\-style causal diagnostic\.

Figure[2](https://arxiv.org/html/2607.13115#S4.F2)reports the necessity test as described\. We can see that removing a small fraction of explainer\-ranked edges produces a much steeper accuracy drop than removing the same number of random edges\. Concretely, the curve indicates that dropping roughly∼8%\\sim 8\\%of top explainer edges yields a degradation comparable to dropping roughly∼25%\\sim 25\\%of randomly chosen edges, illustrating that explainer\-selected edges concentrate decision\-critical information\. The plateau after2525% for both scenarios is the result of most graphs becoming disconnected so the message\-passing regime of the GNN expert breaks down\.

Moving on to the case study, Figure[3](https://arxiv.org/html/2607.13115#S4.F3)illustrates a case where providing the motif and asking the SLM to reason about its chemical implications helps produce a correct label even when theGINexpert is wrong\. This behavior is important for agentic deployments: tools are not always oracle\-correct, and a reliable agent should be able to*cross\-check*tool outputs using additional evidence and domain constraints\. The case study suggests a two\-level governance pattern: the graph expert provides a strong inductive bias, while the SLM serves as a*semantic controller*that can \(i\) interpret explanations, \(ii\) identify inconsistencies, and \(iii\) decide whether to accept or ignore the expert output\. This resembles*human\-in\-the\-loop causal evaluation*, except here the “human” role is approximated by language\-based reasoning conditioned on explicit evidence\.

![Refer to caption](https://arxiv.org/html/2607.13115v1/images/reasoning.png)Figure 3:Example showing that motif\-based reasoning can improve the final prediction \(and, in some cases, correct the GNN expert\) by using the subgraph as structured evidence and reason on its importance\.

## 5Conclusion

We presented a simple, modular approach to improving zero\-shot molecular property prediction with small language models by enabling tool use\. A trained GNN expert provides predictive hints and aGNNExplainer\-derived explanatory motif, which together serve as structured, auditable context that mitigates the structural limitations ofSMILES\-only prompting\. AcrossMUTAGand a subset ofTox21, we find that enriching prompts with expert confidence, subgraph evidence, and motif\-based reasoning can yield substantial gains, particularly on the more challengingTox21setting, while still leaving a gap to specialized graph models\. We further validated that explainer\-selected edges behave like necessary structure for the expert under an intervention\-style edge\-drop test, supporting a causal grounding perspective\. Overall, these results position tool\-augmented SLMs as lightweight agentic systems that combine neural prediction with symbolic graph evidence, offering a practical step toward causally grounded neuro\-symbolic agents in scientific domains\.

## Declaration on Generative AI

During the preparation of this work, the authors used GPT\-5, Gemini 3, and Grammarly for the purposes of: Drafting content, Grammar and spelling check, Paraphrase and reword, Improve writing style\. After using these tools/services, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content\.

## References

## Appendix APrompts Used in Configurations

The following table contains the prompts used for the different scenarios as explained in the Methodology Section[3](https://arxiv.org/html/2607.13115#S3)\. To make sure the SLMs generate valid labels, we validate their output using Pydantic222[https://docs\.pydantic\.dev/latest/](https://docs.pydantic.dev/latest/)templates, allowing only for binary predictions when running an inference setup\. All SLMs are locally hosted using an Ollama333[https://ollama\.com/](https://ollama.com/)server\.

Table 3:Different prompts used with SLMs\. The “Common” and “Task” prompts are used at the start and end of each variant\. The term “Toxic” is substituted with “Mutagenic” when running on the MUTAG dataset\.VariantPromptCommonYou are an expert toxicologist\. Classify the molecule as Toxic \(True\) or Non\-toxic \(False\)\.TaskPredict toxicity based on this SMILES: \{full\_smiles\}\.SMILESCommon \+ TaskSMILES \+ SUBGRAPHCommon \+Critical Subgraph: \{sub\_smiles\}\.\+ TaskSMILES \+ HINTCommon \+An expert model prediction is that it may be \{gnn\_pred\} \(confidence: \{gnn\_conf\} %\)\.\+ TaskSMILES \+ REASONINGCommon \+A possible chemical reasoning for the molecule’s label is: \{chem\_reason\}\.\+ TaskALL CONTEXTCommon \+ All of the above \+ TaskRegarding theChemReasontool that generates the reasoning paragraph for each subgraph, we use the following prompt template for each SLM:

"""

Youareacomputationaltoxicologist\.

Task:Giventhemolecule\{full\_smiles\}andthesubgraph\{sub\_smiles\},

writeONEconciseparagraphexplainingthepotentialchemicalimpactofthesubstructure\.

Ifthesubstructurecontainsknowntoxicophores,explainwhytheyaredangerous\.

IfthesubstructureappearsgenericordoesNOTcontainknowntoxicfeatures,

explicitlystatethatitmaynotbetoxicorthatitsroleisunclear\.

Donotforceatoxicexplanationifonedoesnotexist\.

"""