Model Agnostic Graph Prompt Learning for Crystal Property Prediction

# Model Agnostic Graph Prompt Learning for Crystal Property Prediction
Source: [https://arxiv.org/html/2607.08996](https://arxiv.org/html/2607.08996)
[Shrimon Mukherjee](https://arxiv.org/html/2607.08996v1/mailto:[email protected])School of Mathematical & Computational Sciences Indian Association for the Cultivation of Science Kolkata IndiaEqual Contribution[Kishalay Das](https://arxiv.org/html/2607.08996v1/mailto:[email protected])Department of Computer Science and Engineering Indian Institute of Technology Kharagpur IndiaEqual ContributionPartha BasuchowdhuriSchool of Mathematical & Computational Sciences Indian Association for the Cultivation of Science Kolkata IndiaPawan GoyalNiloy GangulyDepartment of Computer Science and Engineering Indian Institute of Technology Kharagpur India

###### Abstract

Graph Neural Networks have emerged as a powerful tool for the fast and accurate prediction of various crystal properties\. These models often encode domain\-specific knowledge into their graph encoding modules, which increases their parameter size and makes their performance heavily dependent on domain expertise\. Added to this, explicitly incorporating all chemical and structural features, that might influence a specific crystal property into the GNN encoder, is a challenging task\. In this work, we propose a soft prompt learning framework that captures latent features essential for property prediction, which are not explicitly provided to the GNN\. We introduce a novel multilevel graph prompt learning framework comprising both node\-level and graph\-level soft prompts\. At the node level, we capture the local chemical semantics of different atom types, while at the graph level, we encode the global structural symmetry of the crystal graph\. Our proposed prompt learning framework is lightweight and seamlessly integrates with any existing GNN encoder\. Extensive experiments on popular benchmark datasets show that incorporating prompt learning significantly improves \(3% \- 15%\) the performance of state\-of\-the\-art GNN models in crystal property prediction tasks\. Furthermore, the learned soft prompts enable cross\-property knowledge transfer, enhancing prediction performance for properties with limited training data\. Code is available at[https://github\.com/shrimonmuke0202/Prompt\.git](https://github.com/shrimonmuke0202/Prompt.git)

## 1Introduction

Predicting crystal properties remains a significant challenge in materials science\. Unlike molecules, which are often represented as regular graphs, crystal materials are modeled as a 3D point cloud of atoms arranged in a well\-defined shape with a repeating, orderly structure\. Accurate and efficient prediction of crystal properties facilitates the identification of new materials with desired characteristics, reducing the need for extensive physical synthesis or testing and significantly accelerating the discovery process\. A natural approach to modeling crystals involves representing them as graphs\. As a result, numerous studies have focused on developing Graph Neural Network \(GNN\) models for crystal property prediction\. Earlier approachesxie2018crystal,chen2019graph,louis2020graph,Wolverton2020,schmidt2021crystalconstruct a multi\-edge graph from the 3D crystal structure and apply GNN model to encode the neighborhood structural information around an atom\. Subsequently, numerous studies have proposed various GNN architecture variants that integrate specific domain knowledge into the encoder to enhance crystal representation learning\. ALIGNNchoudhary2021atomisticincorporates bond angular information among edges to encode many\-body interactions, Matformeryan2022periodicis designed to be invariant to periodicity to capture repeating patterns explicitly, and PotNetlin2023efficientdirectly models inter\-atomic potentials based on physics principles\. A unifying principle across these models is the assumption that crystal properties, such as formation energy, band gap, total energy, etc\. are inherently linked to the chemical semantics of the constituent atoms and the 3D periodic structure\. Consequently, predicting these properties necessitates capturing both the local chemical semantics at the node\-level and the global 3D structural symmetry at the graph level of the crystal material\. Thus, over the years, state\-of\-the\-art GNN models have progressively improved in encoding these aspects by employing sophisticated architectures resulting in more accurate property predictions but in turn also increasing the number of parameters, leading to longer training and inference times as well as greater computational overhead\. However, explicitly incorporatingallchemical and structural features that might influence a specific crystal property into GNN models poses a significant challenge\. This process requires specialized domain expertise and may involve features that are not yet fully understood or discovered\. As a result, the features utilized in these models, while potentially extensive, cannot be deemed exhaustive, which inherently constrains their effectiveness\. One potential solution is to incorporate latent features that can be learned—both at the local node\-level and the global graph level—into the modeling process\. These learned latent features will account for these missing attributes and further enhance the expressive power of both simpler models like CGCNN and advanced models such as ALIGNN, Matformer, and PotNet\. In this work, we proposesoft or embedding\-based prompt learningas a novel approach to learning latent features\. Building on the success of pixel\-level Visual Prompt \(VP\) techniquesbahng2022exploring,wu2023approximated,wu2022unleashingand the use of learnable soft prompts in NLP taskslester2021power,liu2021p, we introduce a new multi\-level graph prompt learning framework for crystal property prediction\. This framework combines both node\-level \(local\) and graph\-level \(global\) soft prompts\. At node\-level, we learn a set ofkkindependent vectors, with each vector potentially representing a \(latent\) chemical feature\. The impact of these features on a node in the crystal graph is computed using attention scores derived from thosekkvectors and their linear projections\. At the graph level, we design prompt templates that capture key structural features such as crystal geometry and orientation\. Our proposed prompt learning approach is efficient, adding only*0\.32%*extra parameters to Matformer, and is compatible with any existing state\-of\-the\-art GNN encoder\. We perform extensive experiments on two widely used benchmark datasets to evaluate the effectiveness of our proposed prompt learning framework against six state\-of\-the\-art GNN models for the crystal property prediction task\. We show that our proposed prompt learning framework results in substantial performance gains for existing GNN\-based architectures on diverse sets of properties across these benchmark materials datasets\. In specific, we observe that incorporating such lightweight soft prompts enhances the performance of simpler models like CGCNN by 15\.11%, which makes it competitive to advanced models \(more parameters\) such as ALIGNN \(vanilla\)\. Additionally, even for sophisticated models like ALIGNN, Matformer, and PotNet, we observe performance improvements of 5\.52%, 6\.64%, and 3\.15%, respectively using these lightweight soft prompts\. These results highlight the significance and innovation of our proposed framework, which enhances the performance of not only simpler models, but also advanced sophisticated models\. Furthermore, we compare prompt learning with several alternative architectural modification strategies and find prompt learning to be both more effective and lightweight\. We further observe cross\-property knowledge transfer capability of learned soft prompts, which enhance the performance on properties with sparse/limited training data\. Moreover, ablation studies validate the significance of both node\-level and graph\-level prompt learning in effectively capturing the chemical and structural semantics of crystal materials\. We provide the code base in the supplementary material\.

![Refer to caption](https://arxiv.org/html/2607.08996v1/x1.png)Figure 1:\(a\) A periodic crystal structure, featuring a point cloud of atoms arranged in repeating patterns\. \(b\) A magnified view of a unit cell, clearly delineating the lattice vectorsL=\[l1,l2,l3\]L=\[l\_\{1\},l\_\{2\},l\_\{3\}\], highlighting the fundamental building blocks of the crystal’s geometric arrangement\. \(c\) Multigraph representation of the unit cell\.
## 2Preliminaries

### 2\.1Crystal Representation

Crystal materials are viewed as a 3D point cloud of atoms with an orderly repeating pattern\. It can be modeled as a minimalunit cell, containing all constituent atoms in specific coordinates and is repeated infinitely in three\-dimensional space on a regular lattice, resulting in periodic structures as shown in Fig\.[1](https://arxiv.org/html/2607.08996#S1.F1)\(a\) and[1](https://arxiv.org/html/2607.08996#S1.F1)\(b\)\. Given a material withNNnumber of atoms in its unit cell, we can describe the unit cell by two matrices:Atom Type Matrix \(A\)andCoordinate Matrix \(C\)\. Atom Type MatrixA=\[a1,a2,…,aN\]T∈ℝN×k\\textit\{A\}=\[\\textit\{a\}\_\{1\},\\textit\{a\}\_\{2\},\.\.\.,\\textit\{a\}\_\{N\}\]^\{T\}\\in\\mathbb\{R\}^\{N\\times k\}denotes set of atomic type in one hot representation, where k is the maximum number of possible atom types\. Coordinate MatrixC=\[c1,c2,…,cN\]T∈ℝN×3\\textit\{C\}=\[\\textit\{c\}\_\{1\},\\textit\{c\}\_\{2\},\.\.\.,\\textit\{c\}\_\{N\}\]^\{T\}\\in\\mathbb\{R\}^\{N\\times 3\}denotes atomic coordinate positions, whereci∈ℝ3\\textit\{c\}\_\{i\}\\in\\mathbb\{R\}^\{3\}corresponds to coordinates ofit​hi^\{th\}atom in the unit cell\. Further, there is an additionalLattice MatrixL=\[l1,l2,l3\]T∈ℝ3×3\\textit\{L\}=\[\\textit\{l\}\_\{1\},\\textit\{l\}\_\{2\},\\textit\{l\}\_\{3\}\]^\{T\}\\in\\mathbb\{R\}^\{3\\times 3\}, which describes how a unit cell repeats itself in the 3D space towardsl1,l2\\textit\{l\}\_\{1\},\\textit\{l\}\_\{2\}andl3\\textit\{l\}\_\{3\}direction to form the periodic 3D structure of the material\. Formally, a given material can be defined asM=\(A,C,L\)\\mathbf\{\\textit\{M\}\}=\(\\textit\{A\},\\textit\{C\},\\textit\{L\}\)and we can represent its infinite periodic structure as:C^=\{c^i\|c^i=ci\+∑j=13kj​lj\}\\hat\{\\textit\{C\}\}=\\\{\\hat\{c\}\_\{i\}\|\\hat\{c\}\_\{i\}=\\textit\{c\}\_\{i\}\+\\sum\_\{j=1\}^\{3\}k\_\{j\}\{l\}\_\{j\}\\\}andA^=\{a^i\|a^i=ai\}\\hat\{\\textit\{A\}\}=\\\{\\hat\{a\}\_\{i\}\|\\hat\{a\}\_\{i\}=\{a\}\_\{i\}\\\}, wherek1,k2,k3,i∈Z,1≤i≤Nk\_\{1\},k\_\{2\},k\_\{3\},i\\in Z,1\\leq i\\leq N\.

![Refer to caption](https://arxiv.org/html/2607.08996v1/x2.png)Figure 2:The overall framework of our universal prompt learning method\. For node\-level prompting, we use a template that distinguishes between different atom types based on their chemical semantics\. At the graph level, our prompt template captures the inherent crystal structural symmetry in the graph representation\.

## 3Methodology

Given a property\-tagged crystal dataset𝒟t=\{\(𝒢1,y1\),\(𝒢2,y2\)…\\mathcal\{D\}\_\{t\}=\\\{\(\\mathcal\{G\}\_\{1\},y\_\{1\}\),\(\\mathcal\{G\}\_\{2\},y\_\{2\}\)\\dots\(𝒢m,ym\)\}\(\\mathcal\{G\}\_\{m\},y\_\{m\}\)\\\}for property prediction, where crystal structure is represented as multi\-graphs𝒢i=\(𝒱i,ℰi,𝒳i,ℱi\)\\mathcal\{G\}\_\{i\}=\(\\mathcal\{V\}\_\{i\},\\mathcal\{E\}\_\{i\},\\mathcal\{X\}\_\{i\},\\mathcal\{F\}\_\{i\}\), the objective of the property prediction task is to predictyi∈ℝy\_\{i\}\\in\\mathbb\{R\}based on the graph representation of the crystal graphh𝒢ih\_\{\\mathcal\{G\}\_\{i\}\}\. Formally, this involves learning a mapping function:f:𝒢i→yif:\\mathcal\{G\}\_\{i\}\\rightarrow y\_\{i\}\. The conventional approach to model the aforementioned functionffis to use Graph Neural Network \(GNN\) architectures due to their superior ability to encode graph\-structured data\. To further improve prediction performance, we incorporate graph prompt learning into GNN models to learn latent chemical and structural semantics tailored to a specific property\. Next in this section, we first provide a brief background about the crystal property predictor, followed by the proposed prompt learning framework with detailed explanation of node and graph level prompts, and finally, the learning process\.

### 3\.1Crystal Property Predictor

Conventional GNN based crystal property predictor has two building blocks : \(a\) a graph embedding modulefθf\_\{\\theta\}comprising multiple GNN layers, and \(b\) a projection headzϕz\_\{\\phi\}, whereθ\\thetaandϕ\\phiare the parameters of the respective modules\. The graph embedding module takes multi\-edge graph structure along with node and edge features as input and generates a set of node embeddingsℋ\\mathcal\{H\}as output\.

ℋ=\{h1,…,h\|𝒱\|\}=fθ​\(𝒜,𝒳,ℱ\)\\mathcal\{H\}=\\\{h\_\{1\},\.\.\.,h\_\{\|\\mathcal\{V\}\|\}\\\}=f\_\{\\theta\}\(\\mathcal\{A\},\\mathcal\{X\},\\mathcal\{F\}\)\(1\)where𝒜\\mathcal\{A\}is the adjacency matrix of the crystal graph\. Further, a READOUT operation \(symmetric aggregation e\.g\. average, sum, or graph pooling\) is performed to get the representationh𝒢h\_\{\\mathcal\{G\}\}for the whole crystal graph𝒢\\mathcal\{G\}\. This graph representation is then input into a projection headzϕz\_\{\\phi\}, which predicts the property valuey^\\hat\{y\}\.

y^=zϕ​\(h𝒢\);h𝒢=R​E​A​D​O​U​T​\{hu\(L\)\|u∈𝒱\}\\begin\{split\}\\hat\{y\}=z\_\{\\phi\}\(h\_\{\\mathcal\{G\}\}\);\\\>\\\>\\\>h\_\{\\mathcal\{G\}\}=READOUT\\biggl\\\{h\_\{u\}^\{\(L\)\}\|u\\in\\mathcal\{V\}\\biggr\\\}\\end\{split\}\(2\)The entire crystal property predictor architecture, including the GNN layers and the projection head, is trained end\-to\-end to optimize their parameters on downstream property prediction tasks\.

### 3\.2Crystal Prompt Learning

The effectiveness of a GNN\-based crystal property predictor largely depends on its ability to encode the chemical semantics of the constituent atoms and the crystal structural topology into a lower\-dimensional embedding space, thereby generating the graph representationh𝒢h\_\{\\mathcal\{G\}\}\. However, explicitly providing all the chemical and structural semantics that influence a specific crystal property to GNN models is inherently challenging\. Consequently, certain latent features, both at the local node\-level and the global graph level, that affect specific crystal property, are often not captured in the existing models\.

To capture the impact of such latent features, we propose a*Multi\-level Graph Prompt Learning*approach \(Fig\.[2](https://arxiv.org/html/2607.08996#S2.F2)\), introducing learnable soft prompts at both the node and graph levels to capture essential chemical and structural latent features for specific crystal property prediction\. Recently, soft prompts have been effectively utilized to integrate additional meta\-information into input data across various computer vision and NLP tasks\. In computer vision, pixel\-level Visual Prompt \(VP\) techniquesbahng2022exploring,wu2022unleashing,wu2023approximatedmodify input images by adding learnable soft visual prompts, which serve as visual cues to guide the decision\-making of machine learning models\. Similarly, in NLP, researchersliu2021p,lester2021powerhave demonstrated significant potential in incorporating additional meta\-information by introducing learnable soft prompts into sentences, such as adding positional encodings to input tokens to convey positional information\.

Building on these recent successes, in our proposed methodology, we transformed the input graph𝒢\\mathcal\{G\}to prompted graphψ​\(𝒢\)\\psi\{\(\\mathcal\{G\}\)\}by introducing learnable soft prompts into both the feature space \(input node feature\) and latent space \(graph embedding\) of the crystal graph\. We fed the transformed graphψ​\(𝒢\)\\psi\{\(\\mathcal\{G\}\)\}into a GNN model and trained the learnable soft prompts together with the GNN model parameters in an end\-to\-end supervised manner\. In the following subsections, we provide a detailed description of both the node and graph\-level prompting methods\.

#### 3\.2\.1Node\-Level Prompt

Goal of node/atom level prompt learning is to capture untapped chemical attributes \(local properties\), which play a crucial role in predicting a particular crystal property\. To this effect we extend the idea of the universal graph prompt technique proposed byfang2024universaland posit that there arekkdifferent chemical features \(kkis a hyper\-parameter\) which we learn throughkkindependent vectors:b1,b2,\.\.bk∈ℝdb\_\{1\},b\_\{2\},\.\.b\_\{k\}\\in\\mathbb\{R\}^\{d\}andkklinear projectionsa1,a2,\.\.aka\_\{1\},a\_\{2\},\.\.a\_\{k\}\. The learned vectors are utilized through an attention\-based aggregation mechanism to produce node\-level prompts\. First, for each nodeuu, we computekkattention scores leveraging softmax over linear projection of node featurexux\_\{u\}:

αu,j=e​x​p​\(ajT​xu\)∑lke​x​p​\(alT​xu\)\\alpha\_\{u,j\}=\\frac\{exp\(a^\{T\}\_\{j\}x\_\{u\}\)\}\{\\sum^\{k\}\_\{l\}exp\(a^\{T\}\_\{l\}x\_\{u\}\)\}\(3\)Further, we compute prompt vectorpup\_\{u\}through attention\-based aggregation leveraging attention scores and thosekkvectors:

pu=∑jkαu,j​bjp\_\{u\}=\\sum^\{k\}\_\{j\}\\alpha\_\{u,j\}b\_\{j\}\(4\)We apply the same process to each node, generating a unique prompt vector for each one\. These prompt vectors are then combined with their corresponding input node features to create the prompted node feature set𝒳′\\mathcal\{X\}^\{\\prime\}

𝒳′=\{x¯u=xu\+pu\|u∈𝒱\)\}\\mathcal\{X^\{\\prime\}\}=\\\{\\bar\{x\}\_\{u\}=x\_\{u\}\+p\_\{u\}\|u\\in\\mathcal\{V\}\)\\\}\(5\)This prompted node feature set𝒳′\\mathcal\{X\}^\{\\prime\}is subsequently fed into the GNN\-based graph embedding module to generate the graph representationh𝒢h\_\{\\mathcal\{G\}\}for the entire crystal graph𝒢\\mathcal\{G\}\.

ℋ=fθ​\(𝒜,𝒳′,ℱ\);h𝒢=R​E​A​D​O​U​T​\{ℋ\}\\begin\{split\}\\mathcal\{H\}=f\_\{\\theta\}\(\\mathcal\{A\},\\mathcal\{X\}^\{\\prime\},\\mathcal\{F\}\);\\quad h\_\{\\mathcal\{G\}\}=READOUT\\\{\\mathcal\{H\}\\\}\\end\{split\}\(6\)Thesekkvectorsbib\_\{i\}and projection layersaia\_\{i\}are trained on downstream property prediction task \(Equation[8](https://arxiv.org/html/2607.08996#S3.E8)\) for a specific property of interest\. Intuitively, thesekkvectors representkklatent chemical features essential for the property but not explicitly provided to the GNN through the input node features\. Additionally, the projection layers are trained to determine the importance \(attention\) of these latent features for a specific node based on its corresponding node features\. The hyperparameterkkwas found to produce optimal results when set to 10 based on empirical testing\. As a result, the additional parameter overhead to the GNN model is minimal\.

#### 3\.2\.2Graph\-Level Prompt

In addition to the atomic chemical semantics, crystal properties are strongly correlated with the geometric symmetry of crystal structures\. By definition, a crystal material is a periodic arrangement of repeating “motifs”\( e\.g\. atoms, ions\)\. The space group of a crystal describes the symmetry of that crystal which is an important aspect of that crystal’s internal structure\. In turn crystals across certain space groups show similarities among each other, hence they are divided among seven crystal systems: Triclinic, Monoclinic, Orthorhombic, Tetragonal, Trigonal, Hexagonal, and Cubic systemkittel2018kittel\. The goal of graph\-level prompt learning is to design a prompt template that captures underlying crystal geometry in graph representation\. Toward that goal, we train distinct soft prompts for each crystal system and combine them with the graph embeddings\. In specific, we learn an embedding matrix𝐐∈ℝ7×d\\mathbf\{Q\}\\in\\mathbb\{R\}^\{7\\times d\}, containing learnable soft prompts corresponding to each crystal system\. For any specific, crystal graph𝒢\\mathcal\{G\}, with crystal systemτ𝒢\\tau\_\{\\mathcal\{G\}\}, we retrieveτ𝒢\\tau\_\{\\mathcal\{G\}\}row of𝐐\\mathbf\{Q\}and add it as soft promptsqτ𝒢q\_\{\\tau\_\{\\mathcal\{G\}\}\}with the graph embeddingh𝒢h\_\{\\mathcal\{G\}\}\. Formally,

qτ𝒢←𝐐​\[τ𝒢,:\];h¯𝒢=h𝒢\+qτ𝒢\\begin\{split\}q\_\{\\tau\_\{\\mathcal\{G\}\}\}\\leftarrow\\mathbf\{Q\}\[\\tau\_\{\\mathcal\{G\}\},:\];\\quad\\bar\{h\}\_\{\\mathcal\{G\}\}=h\_\{\\mathcal\{G\}\}\+q\_\{\\tau\_\{\\mathcal\{G\}\}\}\\end\{split\}\(7\)The updated graph representation,h¯𝒢\\bar\{h\}\_\{\\mathcal\{G\}\}, is subsequently passed through a projection head,zϕz\_\{\\phi\}, to predict the property values\.

#### 3\.2\.3Prompt Learning Framework

For a specific property prediction task, we convert the input crystal graph into a prompted graph by integrating node\-level prompts with the input node features and graph\-level prompts with the graph embeddings\. This prompted graph is then fed into a state\-of\-the\-art crystal GNN model, which is trained end\-to\-end\. We use mean square error \(MSE\) objective function between predicted𝒴^\\hat\{\\mathcal\{Y\}\}and true property values𝒴\\mathcal\{Y\}

m​i​nθ,ϕ,bi∗,ai∗,qτ𝒢​ℒM​S​E=∥𝒴^−𝒴∥2\\underset\{\\theta,\\phi,b^\{\*\}\_\{i\},a^\{\*\}\_\{i\},q\_\{\\tau\_\{\\mathcal\{G\}\}\}\}\{min\}\\ \\mathcal\{L\}\_\{MSE\}=\\lVert\{\\hat\{\\mathcal\{Y\}\}\}\-\{\\mathcal\{Y\}\}\\rVert^\{2\}\(8\)wherebi∗b^\{\*\}\_\{i\}andai∗a^\{\*\}\_\{i\}are the independent vectors and linear projections for node\-level prompts respectively,qτ𝒢q\_\{\\tau\_\{\\mathcal\{G\}\}\}is the graph\-level prompt andθ\\theta&ϕ\\phiare the trainable parameters of GNN module and projection head respectively\. The proposed prompt learning framework is trained from scratch and optimized for any downstream property prediction task\. All these independent vectorsbi∗b^\{\*\}\_\{i\}, projection layersai∗a^\{\*\}\_\{i\}and graph level promptsqτ𝒢q\_\{\\tau\_\{\\mathcal\{G\}\}\}are randomly initialized and trained along with other model parameters of the GNN property predictor\. We hypothesize that training enables these soft prompt templates to capture intrinsic chemical and structural semantics, which is essential for estimating various crystal properties\.

PropertyCGCNNCGCNNALIGNNALIGNNMatformerMatformerPotNetPotNetiComFormeriComFormereComFormereComFormer\(Prompt\)\(Prompt\)\(Prompt\)\(Prompt\)\(Prompt\)\(Prompt\)\#Parameter0\.423M0\.433M1\.10M1\.11M3\.27M3\.28M0\.473M0\.484M1\.38M1\.39M3\.72M3\.73MFormation Energy0\.0630\.0410\.0350\.0330\.0330\.0320\.02940\.02920\.0270\.0260\.0280\.027Bandgap\(OPT\)0\.2000\.1550\.1420\.1400\.1370\.1360\.12740\.12700\.1350\.1340\.1240\.122Total Energy0\.0780\.0470\.0370\.0350\.0350\.0330\.03240\.03110\.0310\.0290\.0320\.031Ehull0\.1700\.1160\.0760\.0640\.0640\.0530\.05520\.04810\.0470\.0460\.0440\.040Bandgap\(MBJ\)0\.4100\.3540\.3100\.2990\.3000\.2850\.27010\.25800\.2640\.2590\.2800\.276Bulk Modulus\(Kv\)14\.4712\.5410\.4010\.2111\.2110\.0510\.1109\.981014\.2713\.2213\.4512\.26Shear Modulus\(Gv\)11\.7510\.589\.8629\.73110\.769\.5819\.23209\.02019\.7609\.35911\.1210\.56SLME\(%\)5\.6604\.8014\.5214\.4205\.2604\.7644\.57014\.43104\.7024\.3424\.6624\.650Spillage0\.3970\.3880\.3510\.3480\.3980\.3590\.36100\.35220\.3690\.3510\.3820\.369Formation Energy0\.0310\.0290\.0220\.0210\.0210\.0200\.01900\.01830\.0180\.0170\.0180\.016Bandgap\(OPT\)0\.2920\.2880\.2180\.2230\.2110\.2100\.20410\.19420\.1930\.1900\.2020\.199Bulk Modulus\(Kv\)0\.0470\.0430\.0510\.0380\.0430\.0420\.04000\.03820\.0380\.0370\.0420\.039Shear Modulus\(Gv\)0\.0770\.0770\.0780\.0690\.0730\.0670\.06540\.06500\.0640\.0630\.0730\.072

Table 1:Summary of the results \(MAE\) of different properties in JARVIS\-DFT \(top\) and Materials Project \(bottom\)\. Model M is the vanilla variant of the SOTA model and M \(Prompt\) is a prompt\-infused variant \(both node\- and graph\-level\) of the SOTA model\. The best performance has been highlighted inbold\.

## 4Experimental Setup

To assess the effectiveness of our proposed graph prompt learning framework, we perform experiments using two benchmark crystal datasets: Materials ProjectMPand JARVIS\-DFTchoudhary2020joint\. We choosesix popular state\-of\-the\-art GNN modelsfor supervised crystal property prediction; these are CGCNNxie2018crystal, ALIGNNchoudhary2021atomistic, Matformeryan2022periodic, PotNetlin2023efficient, iComFormeryan2024complete, and eComFormeryan2024complete\. For a thorough evaluation, we examine both the vanilla variant and the prompt\-infused variant of these SOTA models \(Section[5](https://arxiv.org/html/2607.08996#S5)\)\. To implement prompt\-infused variants, we used the available official PyTorch implementations of all the baselines and incorporated node\- and graph\-level prompts into it\. We train each prompt\-infused model for 1000 epochs using AdamWloshchilov2017decoupledoptimizer with normalized weight decay of10−510^\{\-5\}and keep the batch size as 64\. We schedule the learning rate according to the one\-cycle policysmith2018disciplinedwith a maximum learning rate of 0\.001\. We keep our atom\-type prompt dimension the same as the embedding dimension of the atom and for graph\-level prompt we keep the dimension the same as the embedding dimension of the graphs\. We perform all the experiments on shared servers with NVIDIA A6000 GPUs\.

## 5Experimental Results

In this section, we first evaluate the performance of our proposed prompt learning framework against baseline models using benchmark datasets and provide a detailed analysis \(Section[5\.1](https://arxiv.org/html/2607.08996#S5.SS1)\)\. Further, we asses the performance of the prompt learning model in conjunction with an existing pre\-trained model and compare it against well\-established methods, including graph prompt tuning and conventional fine\-tuning approaches \(Section[5\.2](https://arxiv.org/html/2607.08996#S5.SS2)\)\. Next, we compare the effectiveness of prompt learning with alternative architectural modifications \(Section[5\.3](https://arxiv.org/html/2607.08996#S5.SS3)\)\. Additionally, we demonstrate the cross\-property knowledge transfer capability of learned prompts in limited data scenarios \(Section[5\.4](https://arxiv.org/html/2607.08996#S5.SS4)\)\. Finally, we conduct ablation studies to analyze the impact of different prompt modalities \(Section[5\.6](https://arxiv.org/html/2607.08996#S5.SS6)\) and sensitivity on the number of basis vectors \(Section[5\.7](https://arxiv.org/html/2607.08996#S5.SS7)\)\.

PropertyCrysGNNCrysGNNCrysGNNCrysGNNCrysGNNCrystalCrystal TwinsCrystal TwinsCrystal TwinsCrystal TwinsDPFDPFDPFDPFDPF\(GPF\)\(SUPT\)\(LEAP\)\(Prompt\)Twins\(GPF\)\(SUPT\)\(LEAP\)\(Prompt\)\(GPF\)\(SUPT\)\(LEAP\)\(Prompt\)Formation Energy0\.1460\.7380\.2220\.3220\.1330\.0440\.2450\.1990\.3220\.0410\.0290\.0990\.1480\.1840\.027Bandgap\(OPT\)0\.3250\.8100\.4260\.5200\.3100\.1640\.4800\.3920\.5360\.1510\.1230\.2770\.3270\.4010\.120Total Energy0\.1961\.5030\.4790\.7290\.1640\.0510\.3990\.2550\.3660\.0450\.0320\.1040\.1820\.2210\.029Ehull0\.2270\.9790\.4390\.7250\.2050\.1340\.3640\.2830\.6370\.1190\.0630\.1800\.4320\.1250\.054Bandgap\(MBJ\)0\.6011\.3510\.7120\.9990\.5870\.3790\.8490\.6950\.9960\.3370\.3160\.5290\.5900\.6250\.287Bulk Modulus\(Kv\)17\.3749\.4919\.3635\.2316\.7913\.0814\.8318\.7729\.3912\.4410\.4316\.5718\.9912\.1910\.30Shear Modulus\(Gv\)13\.1925\.2114\.0221\.2213\.1210\.5519\.9214\.8820\.2110\.169\.59613\.6214\.1710\.359\.498SLME\(%\)6\.19210\.456\.4039\.2536\.0504\.8917\.4206\.6498\.2294\.5905\.1296\.4286\.7865\.7454\.724Spillage0\.4190\.5070\.4420\.4930\.4180\.3860\.4320\.4150\.4720\.3830\.3580\.4250\.4360\.5180\.347

Table 2:The performance between finetuned and the prompt\-infused versions of the existing pre\-trained models on nine materials properties from JARVIS\-DFT dataset\. The best performance has been highlighted inbold\.### 5\.1Performance on the benchmark datasets

We choose six SOTA models for supervised crystal property prediction, CGCNN, ALIGNN, Matformer, PotNet, iComFormer, and eComFormer, where we compare the performance of the vanilla variant \(version reported in the respective papers\) of each SOTA model with our proposed prompt\-infused variant\. To avoid any deterioration of the performance due to insufficient hyperparameter tuning, we report the property prediction results for vanilla variants from the respective papers\.

#### 5\.1\.1JARVIS\-DFT Dataset\.

We first conduct experiments on the JARVIS\-DFT dataset, which is a widely used large\-scale material benchmark containing 55,722 crystals\. JARVIS \(Joint Automated Repository for Various Integrated Simulations\)111https://jarvis\.nist\.gov/is a free and publicly available database that includes crystal structures and corresponding properties from DFT\-based calculations, classical force\-fields and machine learning approacheschoudhary2020joint\. In line with previous state\-of\-the\-art studies, we select nine crystal properties for the downstream property prediction task: formation energy, bandgap \(OPT\), bandgap \(MBJ\), total energy, bulk moduli, shear moduli, ehull, spillage, and SLME\(%\)\. We use 80%, 10%, and 10% train, validation and test split for all the properties as used by PotNet\. We report the mean absolute error \(MAE\) of the predicted and actual value of a particular property for test data in Table[1](https://arxiv.org/html/2607.08996#S3.T1)to compare the performance of different participating methods\. We observe that our proposed prompt infused variant of any SOTA model outperforms the vanilla model across all the properties\. In specific, the average improvement in CGCNN, ALIGNN, Matformer, PotNet, iComFormer, and eComFormer are 20\.37%, 4\.23%, 8\.02%, 3\.49%, 4\.32%, and 4\.04% respectively\.

#### 5\.1\.2Materials Project \(MP\) Dataset\.

We further use another benchmark material dataset, Materials Project\-2018\.6\.1, which comprises 69,239 materials\. Materials Project is also a free and publicly available database that contains crystal structures and the corresponding calculated materials propertiesMP\. The dataset is created by taking the results obtained with density functional theory \(DFT\) based calculations\. The dataset consists of data on electronic structure, as well as thermodynamic, mechanical, and dielectric characteristics\. Additionally, it offers a graphical web\-based search platform222https://materialsproject\.org/\. Following the previous SOTA algorithms, here we choose four crystal properties, namely, formation energy, bandgap \(OPT\), bulk modulus \(Kv\), and shear modulus \(Gv\), respectively\. We follow the previous line of work to split the data for training, validation, and test\. For formation energy and bandgap\(OPT\), we use 60,000, 5,000, and 4,239 crystals as train, validation, and test split as used by ALIGNN, whereas we use 4,664, 393, and 393 crystals as train, validation, and test split for bulk and shear moduli as used by Matformer\. In Table[1](https://arxiv.org/html/2607.08996#S3.T1), we report the MAE of different baseline models and their prompt\-infused variants on the MP dataset\. To ensure consistency with the results reported by baseline studies, we present bulk and shear moduli values in GPa for JARVIS\-DFT and log\(GPa\) values for MP\. Our findings indicate that the prompt\-infused variant of any SOTA model consistently outperforms the standard model across nearly all properties in the MP dataset\. In specific, the average improvement in CGCNN, ALIGNN, Matformer, PotNet, iComFormer, and eComFormer are 4\.08%, 9\.82%, 3\.94%, 3\.41%, 2\.83%, and 5\.28% respectively\. Additionally, the impressive performance of all the prompt infused variants on the Bulk Moduli and Shear Moduli tasks with just 4,664 training samples highlights the robustness and adaptability of our prompt learning framework in scenarios with limited training data\. Overall, the performance improvements observed across both datasets and various SOTA GNN models demonstrate the effectiveness of our proposed prompt learning framework in capturing latent chemical and structural semantics, thereby generating more enriched crystal representations\. Historically, the research community has focused on developing more complex GNN architectures to integrate intrinsic domain knowledge into the encoder, resulting in a significant increase in the number of parameters \(see the \#Parameters row in Table[1](https://arxiv.org/html/2607.08996#S3.T1)\)\. In contrast, our proposed prompt\-tuning framework introduces approximately*10k*additional parameters across all models as node\- and graph\-level soft prompts\. This addition is negligible but substantially improves the performance of simpler models like CGCNN, making them competitive with more advanced models such as ALIGNN, Matformer, PotNet, iComFormer, and eComFormer\.

### 5\.2Comparison with Pre\-trained Models

In this section, we investigate whether our proposed prompt learning framework helps us to enhance existing crystal pre\-trained models in property prediction performance\. Specifically, we compare the prompt\-infused variants of these pre\-trained models against both conventional graph prompt learning approaches and traditional fine\-tuning methods applied to the same crystal pre\-trained models\. We take CrysGNNdas2023crysgnn, Crystal Twinsmagar2022crystal, and DPFshen2025denoisingas the pre\-trained models\. CrysGNN and Crystal Twins are initially pre\-trained using a self\-supervised framework, whereas DPF leverages diffusion\-based pre\-training on a large corpus of property\-unlabeled data before being fine\-tuned for downstream property prediction tasks\. Note that the pre\-training frameworks of CrysGNN and Crystal Twins are built upon CGCNN, whereas DPF adopts Matformer as its underlying graph embedding module\. We adopt GPFfang2024universal, SUPTlee2024subgraph, and LEAPxu2025learningas state\-of\-the\-art graph prompt learning baselines, where the pre\-trained models are kept frozen and only graph prompts are tuned\. For our prompt\-infused variants, we begin with a crystal pre\-trained model, integrate both node\-level and graph\-level prompts, and subsequently fine\-tune the model, along with soft prompts, on a specific crystal property prediction task\.

GNNModelSetup\# ParametersFormationEnergyBandgap\(OPT\)TotalEnergyEhullVanilla0\.423M0\.0630\.2000\.0780\.1702×\\timesGNN Layers0\.788M0\.0610\.1740\.0730\.162Increase Hidden Dim1\.435M0\.0550\.1770\.0720\.175CGCNNSkip Connection0\.423M0\.0590\.1920\.0730\.161Adapter Learning0\.476M0\.1510\.2240\.2780\.261Prompt Learning0\.433M0\.0410\.1550\.0470\.116Vanilla3\.27M0\.0330\.1370\.0350\.0642×\\timesGNN Layers6\.34M0\.0320\.1430\.0350\.057Increase Hidden Dim12\.84M0\.0520\.1390\.0920\.104MatformerSkip Connection3\.27M0\.0330\.1380\.0340\.064Adapter Learning3\.36M0\.0890\.1560\.1940\.155Prompt Learning3\.28M0\.0320\.1360\.0330\.053

Table 3:Summary of results \(MAE\) for various properties in JARVIS\-DFT using different architectural modifications in CGCNN and Matformer\. Our proposed prompt learning paradigm is both lightweight and delivers superior performance compared to alternative approaches\.We report the results of all competing models for nine properties formation energy, bandgap\(OPT\), total energy, ehull, bandgap\(MBJ\), bulk moduli\(Kv\), shear moduli\(Gv\), SLME, and spillage from JARVIS dataset in Table[2](https://arxiv.org/html/2607.08996#S5.T2)\. We observe that the prompt\-tuned variants of these pre\-trained models outperform both conventional graph prompt learning as well as fine\-tuned versions by a good margin for all nine properties\. Specifically, compared to fine\-tuned versions the average improvements in CrysGNN, Crystal Twins, DPF are 5\.36%, 7\.14%, and 6\.13% respectively\. This demonstrates that incorporating prompts into pre\-trained models and fine\-tuning enhances the model’s predictive performance compared to the traditional fine\-tuning framework\. However, when employing a conventional graph prompt learning framework—while keeping the pre\-trained model weights frozen—we observe negative transfer and a noticeable degradation in performance across all evaluated properties\. One possible reason is that, unlike other graph domains such as molecular structures, the pre\-trained models for crystal structures are less powerful and have been trained on relatively limited data\. Additionally, existing graph prompt\-tuning methods tend to overlook unique characteristics inherent to crystal structures, such as crystal symmetry and other global features\. Finally, prior prompt\-based methods such as GPF, SUPT, and LEAP have primarily been evaluated on binary classification tasks in molecular datasets, whereas our work focuses on regression\-based crystal property prediction, which poses a more challenging learning problem\.

Prompt TransferPropertiesVanillaPromptLearningFormationEnergyBandgap\(OPT\)TotalEnergybg\(MBJ\)0\.4100\.3540\.3370\.3520\.340Shear11\.7510\.5810\.6810\.3810\.40Bulk14\.4712\.5412\.0612\.4312\.42SLME5\.664\.814\.684\.944\.70Spillage0\.3970\.3880\.3840\.3830\.380

Table 4:Results showing the knowledge transfer capability of soft prompts, trained on properties with more training data and applied to properties with limited training data\.
### 5\.3Alternate Architectural Modifications

Next, we explore alternative methods for learning latent chemical and structural features by making simple modifications to the existing GNN models\. The approaches investigated are increasing hidden layer dimensions, doubling the number of GNN layers, introducing skip connections, and implementing adapter learning\.

1. 1\.2×\\timesGNN Layers\.In this variant, the number of GNN layers is increased by doubling the original number of layers\.
2. 2\.Increasing Hidden Dim\.In this variant, the hidden layer dimensions for each GNN are increased by 64\.
3. 3\.Skip Connection\.In this variant, residual connections are added to each layer of the GNN\.
4. 4\.Adapter Learning\.In this variant, a parameter\-efficient adapter\-based tuning technique is used for each layer of the GNN\. The method follows the approach proposed in AdapterGNNli2024adaptergnn, where parameter\-efficient adapters are added to each layer of the GNN models\.

We conduct another set of experiments using JARVIS\-DFT dataset on CGCNN and Matformer to compare the performance of these alternative methods with prompt learning and report the results in Table[3](https://arxiv.org/html/2607.08996#S5.T3)and Table[6](https://arxiv.org/html/2607.08996#A4.T6)\. We observe that compared to all other approaches, prompt learning introduces minimal parameter overhead while outperforming all of these\. In particular, increasing hidden layer dimensions or doubling the number of GNN layers substantially increases the model’s parameter count, yet results in minimal performance improvement across all properties\. On the other hand, while adapter learning and skip connections add only a small parameter overhead, they also fail to significantly enhance crystal property prediction performance\. This highlights the ingenuity of our graph prompt learning approach, which is lightweight, model\-agnostic, and provide substantial improvement over the vanilla models\.

### 5\.4Cross\-Property Knowledge Transfer Capability of Soft Prompts

A major challenge in developing crystal property predictor is the lack of labeled datasets for all properties\. For instance, in the JARVIS DFT dataset, properties such as formation energy or bandgap \(OPT\) typically have larger training data, while properties such as bulk moduli, shear moduli, or bandgap \(MBJ\) have limited data available\. This raises an interesting question:*Can the knowledge captured by soft prompts for well\-represented properties be transferred to improve performance on under\-represented properties with limited data?*To explore this further, we designed an additional experiment on JARVIS\-DFT dataset where soft prompts \(both node\-level and graph\-level\) are trained on one of the properties with abundant training data like formation energy, bandgap\(OPT\), or total energy\. While training other properties having limited data, instead of randomly initializing the soft prompts, we initialize using the learned prompts\. We report the results of four properties in Table[4](https://arxiv.org/html/2607.08996#S5.T4)\. We observe that when a property with limited training data is correlated with a property that has abundant training data, the soft prompts learned from the latter can improve the performance of the former\. For instance, properties like bandgap \(MBJ\), bulk modulus, and SLME are known to correlate with energies such as formation energy or total energy\. Therefore, we observe that transferring knowledge from soft prompts learned on formation energy or total energy further enhances the performance of bandgap \(MBJ\), bulk modulus, and SLME compared to the conventional prompt learning approach\. In contrast, shear modulus is unrelated to formation energy, and SLME is not correlated with bandgap \(OPT\), leading to slight degradation in performance\. Overall, these results highlight the knowledge transfer capabilities of learned soft prompts and enhance model performance in limited data settings\.

![Refer to caption](https://arxiv.org/html/2607.08996v1/x3.png)Figure 3:Periodic table visualization of the mean node\-level prompt embedding values for dimension 7 across elements\. Tile colors denote prompt values, while the numbers indicate the original covalent radius \(pm\)\.
### 5\.5Expressiveness of Node\-Level Soft Prompt

To evaluate the expressiveness of the learned node\-level soft prompts, we examine whether individual prompt dimensions capture meaningful periodic trends across elements\. For each element, we compute the mean value of the corresponding prompt embedding across all occurrences in the dataset and visualize the resulting distribution over the periodic table\. Interestingly, we observe that certain prompt dimensions align with chemically meaningful elemental properties\. As shown in Fig\.[3](https://arxiv.org/html/2607.08996#S5.F3), prompt embedding dimension seven exhibits clear periodic patterns across elements and shows a strong positive correlation with covalent radius \(r=r=0\.8095\)\. In particular, electropositive elements with large covalent radii—such as alkali metals \(e\.g\., K, Rb, Cs\), alkaline\-earth metals \(e\.g\., Ca, Sr, Ba\), and lanthanides—tend to exhibit relatively higher prompt values, whereas smaller non\-metallic elements such as C, N, O, and F generally show lower values\. This pattern broadly follows the periodic variation of covalent radiuscordero2008covalent,pyykk02015additive, suggesting that the learned prompt embeddings implicitly encode atomic\-size\-related information relevant to bonding environments in crystalline materials\. Additional prompt dimensions also show correlations with other elemental descriptors, including electronegativity and ionization energy\. A detailed correlation analysis between prompt dimensions and elemental properties is provided in Appendix[D\.8](https://arxiv.org/html/2607.08996#A4.SS8)\. For additional analysis, we visualize the node\-level prompt embeddings using t\-SNEvan2008visualizing\(Fig\.[7](https://arxiv.org/html/2607.08996#A4.F7)\), coloring elements by their periodic block\. The resulting embedding space exhibits clear structure: f\-block elements form a compact, well\-separated cluster; s\-block elements appear as distinct outliers; and d\- and p\-block elements occupy broader, continuous regions with partial overlap\. This organization aligns with known chemical principles, where certain element groups are sharply distinguished while others exhibit more gradual transitions\. To quantify this structure, we perform k\-means clustering on the prompt embeddings, obtaining a clustering purity of 0\.73 and a Normalized Mutual Information \(NMI\) of 0\.64 against ground\-truth periodic\-block labels\. Together, these qualitative and quantitative results provide complementary evidence that the learned prompt embeddings capture chemically meaningful relationships beyond simple correlation\.

EncoderStrategyTuningStrategyFormationEnergyBandgap\(OPT\)TotalEnergyEhullCGCNNVanilla0\.0630\.2000\.0780\.170Only Node0\.0410\.1600\.0480\.117Only Graph0\.0420\.1560\.0490\.119Both0\.0410\.1550\.0470\.116ALIGNNVanilla0\.0350\.1420\.0370\.076Only Node0\.0340\.1410\.0350\.066Only Graph0\.0340\.1410\.0360\.065Both0\.0330\.1400\.0350\.064PotNetVanilla0\.02940\.1270\.03210\.055Only Node0\.02940\.1270\.03170\.053Only Graph0\.02990\.1290\.03140\.049Both0\.02920\.1270\.03100\.048

Table 5:Ablation results with different modalities of prompt learning strategies\. Best results isboldand the second best result isunderlined\.![Refer to caption](https://arxiv.org/html/2607.08996v1/x4.png)Figure 4:This illustration shows the sensitivity analysis on the choice of independent basis vectors on four properties \(a\) Bandgap\(OPT\), \(b\) Ehull, \(c\) Bulk Modulus\(Kv\), and \(d\) Shear Modulus\(Gv\) using JARVIS\-DFT dataset
### 5\.6Impact of Different Prompt Modalities

We conduct an ablation study to investigate the influence of different prompt modalities in enhancing SOTA model performance and report the results in Table[5](https://arxiv.org/html/2607.08996#S5.T5)\. Specifically, we conduct experiments with three setups: 1\) By default, in our proposed framework we infused both node and graph\-level prompts \(denoted as*“Both"*\), 2\) With only node\-level prompts infused at input node\-features and no graph\-level prompts \(denoted as*“Only Node"*\), 3\) With only graph\-level prompts infused at graph embedding and no node\-level prompts \(denoted as*“Only Graph"*\)\. For this study, we use three baseline models: CGCNN, ALIGNN, and PotNet, each trained on four properties from the JARVIS\-DFT dataset: formation energy, bandgap \(OPT\), total energy, and Ehull\. It is evident that using both node\-level and graph\-level prompt learning together \(Both\), CGCNN, ALIGNN, and PotNet yields the best results in most cases, surpassing the performance of using node\-level or graph\-level prompt learning alone\. This indicates that integrating both node\-level and graph\-level prompts effectively captures atomic chemical semantics and crystal topological information, leading to enhanced property prediction performance\. Additional results for other properties can be found in the Appendix[D\.2](https://arxiv.org/html/2607.08996#A4.SS2)\.

### 5\.7Sensitivity Analysis on No\. Basis Vectors\.

We also conducted a sensitivity analysis on

kk\(the number of independent basis vectors\) used in node\-prompt learning \(Section[3\.2\.1](https://arxiv.org/html/2607.08996#S3.SS2.SSS1)\)\. For this, we performed an ablation study using the prompt fused variant of CGCNN and PotNet on four properties from the JARVIS\-DFT dataset\. Experiments were conducted with varying values of

kk, and the results are presented in Fig\.[4](https://arxiv.org/html/2607.08996#S5.F4)\. From the figure, we observe that the best MAE for all properties is achieved with

k=10k=10\. However, increasing

kkto 15 or 20 does not yield optimal results\. Consequently, we selected 10 independent basis vectors for node\-level prompts in all our experiments, as presented before\.

Additional Results\.Appendix \([D](https://arxiv.org/html/2607.08996#A4)\) provides details on additional experiments, including a alternative architectural modifications \([D\.1](https://arxiv.org/html/2607.08996#A4.SS1)\), impact of different prompt modalities \([D\.2](https://arxiv.org/html/2607.08996#A4.SS2)\), statistical significance of the results \([D\.3](https://arxiv.org/html/2607.08996#A4.SS3)\), efficiency analysis \([D\.4](https://arxiv.org/html/2607.08996#A4.SS4)\), robustness to label noise[D\.5](https://arxiv.org/html/2607.08996#A4.SS5), contribution of graph\-level prompt[D\.6](https://arxiv.org/html/2607.08996#A4.SS6), fairness of baseline prompt methods[D\.7](https://arxiv.org/html/2607.08996#A4.SS7), expressiveness of node\-level prompt[D\.8](https://arxiv.org/html/2607.08996#A4.SS8)and expressiveness of graph\-level prompt \([D\.9](https://arxiv.org/html/2607.08996#A4.SS9)\)\.

## 6Conclusion

Crystal property prediction remains a challenging task in materials science and recent research focused on encoding intrinsic domain knowledge into GNN encoders by developing increasingly complex architectures\. However, this approach often leads to a significant increase in parameters, resulting in computational overheads\. In this work, we introduce soft prompt learning, a novel paradigm designed to capture latent atomic and structural features in GNN models, thereby enhancing crystal property prediction performance but with minimal increase in parameter space\. Our method incorporates learnable prompts at both the node and graph levels\. At the node level, soft prompts are tailored to capture the local chemical semantics of constituent atoms, while at the graph level, they encode the underlying crystal symmetry\. Through extensive experiments on benchmark datasets, we demonstrate that our framework significantly improves performance across various SOTA GNN models\. Notably, we observe that incorporating soft prompts enhances the expressive power of both simpler models like CGCNN and advanced models such as ALIGNN, Matformer, and PotNet\. Furthermore, when compared to alternative architectural modification strategies, prompt learning proves to be both more effective and lightweight\. Additionally, we show that soft prompts possess cross\-property knowledge transfer capabilities, enabling improved performance in limited data scenarios by transferring learned knowledge across different properties\. We believe that this soft prompt learning paradigm for GNNs has the potential to extend beyond the crystal domain, enhancing the performance of state\-of\-the\-art GNN models in a wide range of downstream tasks\.

## References

Model Agnostic Graph Prompt Learning for Crystal Property Prediction \(Supplementary Material\)

## Appendix AMulti\-graph Construction for Crystals

We realize a crystal material as a multi\-graph structure𝒢=\(𝒱,ℰ,𝒳,ℱ\)\\mathcal\{G\}=\(\\mathcal\{V\},\\mathcal\{E\},\\mathcal\{X\},\\mathcal\{F\}\)as shown in Fig\.[1](https://arxiv.org/html/2607.08996#S1.F1)\(c\)xie2018crystal\.𝒢\\mathcal\{G\}is an undirected weighted multi\-graph where𝒱\\mathcal\{V\}denotes the set of nodes or atoms present in a unit cell of the crystal structure\.ℰ=\{\(u,v,ku​v\)\}\\mathcal\{E\}=\\\{\(u,v,k\_\{uv\}\)\\\}denotes a multi\-set of node pairs andku​vk\_\{uv\}denotes number of edges between a node pair\(u,v\)\(u,v\)\.𝒳=\{\(xu\|u∈𝒱\)\}\\mathcal\{X\}=\\\{\(x\_\{u\}\|u\\in\\mathcal\{V\}\)\\\}denotes 92 dimensional node feature set proposed by CGCNNxie2018crystal\. It includes different chemical properties like electronegativity, valence electron, covalent radius, etc\. Finally,ℱ=\{\{sk\}\(u,v\)\|\(u,v\)∈ℰ,k∈\{1\.\.ku​v\}\}\\mathcal\{F\}=\\\{\\\{s^\{k\}\\\}\_\{\(u,v\)\}\|\(u,v\)\\in\\mathcal\{E\},k\\in\\\{1\.\.k\_\{uv\}\\\}\\\}denotes the multi\-set of edge weights wheresks^\{k\}corresponds to thekt​hk^\{th\}bond length between a node pair\(u,v\)\(u,v\), which signifies the inter\-atomic bond distance between two atoms\.

## Appendix BGNN for Crystal Materials

Graph neural network models have been very effective in learning crystal representation\. CGCNNxie2018crystalis the first proposed model, which represents 3D crystal structure as an undirected weighted multi\-edge graph and applies a graph convolution neural network to update node features based on their local chemical and structural environment\. The fundamental principle of a GNN model is the message passing and aggregation paradigm, where it learns node representations by recursively aggregating the representations \(messages\) from neighboring nodes within the graph\. Formally, atlt​hl^\{th\}layer of a GNN model, node representation of nodeuuis updated as follows :

hu\(l\+1\)=C​O​M​B​\{hu\(l\),A​G​Gv∈Nu​\(hv\(l\),hu\(l\),e\(u,v\)\(l\)k\)\}h^\{\(l\+1\)\}\_\{u\}=COMB\\biggl\\\{h^\{\(l\)\}\_\{u\},\\underset\{v\\in N\_\{u\}\}\{AGG\}\\biggl\(h^\{\(l\)\}\_\{v\},h^\{\(l\)\}\_\{u\},\{e^\{\(l\)\}\_\{\(u,v\)\}\}\_\{k\}\\biggr\)\\biggr\\\}\(9\)wherehu\(l\),hv\(l\)h^\{\(l\)\}\_\{u\},h^\{\(l\)\}\_\{v\}denote node representation of nodesuuandvv, respectively, atlt​hl^\{th\}layer,e\(u,v\)\(l\)k\{e^\{\(l\)\}\_\{\(u,v\)\}\}\_\{k\}denotes representation ofkt​hk^\{th\}edge between\(u,v\)\(u,v\)atlt​hl^\{th\}layer andNuN\_\{u\}denotes the set of neighboring nodes of nodeuu\. The AGG function aggregates messages from the neighboring nodes ofuu, while the COMB function combines this aggregated message with the node representation ofuuto produce the node representation ofuufor the next\(l\+1\)t​h\(l\+1\)^\{th\}layer\. After applyingLLsuch layers of aggregation and combining, the final set of node embeddingsℋ=\{h1,…,h\|𝒱\|\}\\mathcal\{H\}=\\\{h\_\{1\},\.\.\.,h\_\{\|\\mathcal\{V\}\|\}\\\}is produced, wherehu:=hu\(L\)∈ℝdh\_\{u\}:=h\_\{u\}^\{\(L\)\}\\in\\mathbb\{R\}^\{d\}represents the final embedding of nodeuu\.

## Appendix CRelated Work

### C\.1Crystal Property Prediction

Historically, Density Functional Theory \(DFT\)orio2009densityhas been an effective tool for both crystal material generation and estimating several properties of crystals by examining the electronic density of their atomic configurations\. However, DFT calculation requires substantial computation costs; hence, it poses a significant challenge for novel materials discovery in the vast materials space\. Recently, thanks to the advances of deep learning, data\-drivenseko2015prediction,pilania2015structure,lee2016prediction,de2016statistical,seko2017representation,isayev2017universal,ward2017including,lu2018accelerated,im2019identifyingtechniques have become quite popular to establish relationships between crystal atomic structure and their properties effectively\.

Particularly, for material generation, equivariant diffusion modelsxie2021crystal,luo2023towards,jiao2023crystal,yang2023scalable,jiao2024space,miller2024flowmm, Flow Modelslipman2023flow,albergo2023building,liu2023flow,das2025periodicand Large Language Modelgruver2024fine,khastagir2026llm,govindarajan2026crystunebeen explored\. On the contrary, graph neural network \(GNN\)xie2018crystal,chen2019graph,louis2020graph,Wolverton2020,schmidt2021crystal,choudhary2021atomistic,yan2022periodic,lin2023efficientbased approaches become popular for crystal property prediction to encode complex crystal structures along with atom features in an enriched representation space\.

Earlier approachesxie2018crystal,chen2019graph,louis2020graph,Wolverton2020,schmidt2021crystalconstruct a multi\-edge graph from the 3D crystal structure and apply a GNN model to encode the neighborhood structural information around an atom\. Building on this, numerous studies have proposed various GNN architecture variants that integrate domain\-specific knowledge into the encoder to improve crystal representation learning\. ALIGNNchoudhary2021atomisticincorporates bond angular information among edges to capture many\-body interactions; Matformeryan2022periodicis designed to be invariant to periodicity, enabling it to explicitly capture repeating patterns; and PotNetlin2023efficientmodels inter\-atomic potentials directly based on physical principles\. In a related direction, CrysMMNet\[das2023crysmmnet\]predicts material properties by fusing textual descriptions with graph\-based structural representations\. Moreover, data scarcity is a known challenge in this field, and to address it, several graph pre\-training strategies have been introduced\. CrysXPPdas2022crysxppis the first pre\-trained/fine\-tune model, which uses an autoencoder to train on a volume of un\-tagged crystal graphs \(unsupervised pre\-training\) and the learned knowledge is used to initialize the encoder of a property predictor, which is further fine\-tuned with property\-specific tagged data\. Following this work, CrysGNNdas2023crysgnnproposed large\-scale pre\-training with a huge corpus of curated unlabelled data and incorporates the idea of knowledge distillation to distill the important information from the pre\-trained model and injects it into the property prediction process\. Building on this line of work, Crystal Twins\[magar2022crystal\]takes a self\-supervised route, using a Barlow Twins\-style objective to pre\-train a CGCNN backbone\. A separate thread of research turns to diffusion models for learning crystal representations through structure reconstruction\. CrysDiff\[song2024diffusion\], for instance, pre\-trains a joint denoising diffusion model that reconstructs crystal geometry starting only from atomic composition; at finetuning time, the structure is held fixed and the diffusion process is instead conditioned on the target property\. DPF\[shen2025denoising\]follows a related but distinct recipe: it corrupts atom identities, atomic positions, and lattice parameters during pre\-training and trains the model to recover the original structure, after which the resulting representations are finetuned for property\-prediction tasks\. CrysAtom\[mukherjee2025crysatom\]takes yet another approach, learning distributed atom\-level embeddings from unlabeled crystals in an unsupervised manner and shows this pretraining strategy meaningfully improves downstream property\-prediction performance\. Most recently, CrysLDNet\[mukherjee2026latent\]departs from operating directly in the raw feature space and instead moves pretraining into a compact latent space: a variational autoencoder first compresses the crystal structure into a smooth latent representation, and a diffusion model is then trained to denoise within that space rather than over atom types, coordinates, and lattice parameters directly\. The pretrained encoder is subsequently finetuned with a lightweight prediction head for downstream property prediction\.

### C\.2Graph\-Prompt Tuning Methods

Prompt tuning methods, originating from Natural Language Processing \(NLP\), have been widely used to facilitate the adaptation of pre\-trained language models to various downstream tasksliu2021pre\. Particularly, research has shown using soft prompts model can achieve optimal performance in various downstream tasksliu2021pre,lester2021power\. Since, there are widely available pre\-trained models like BERTdevlin2018bert, GPTradford2019language, etc in NLP, traditional prompt\-based approachesliu2021pre,lester2021powerfreeze the parameters of the pre\-train model and introduce additional learnable components in the input space, thereby enhancing the compatibility between inputs and pre\-trained models\. Further, in computer vision, Jia et al\.jia2022visualand Bahng et\. al\.bahng2022exploringinvestigate the efficacy of adapting large\-scale methods by modifying input images at the pixel level\. Inspired by this, few recent studies have explored prompt\-tuning methods for graph neural networks\(GNNs\)\. GPPTsun2022gpptproposes the learnable edge\-level prompts, transforming node classification into a link prediction task to mitigate the structural gap\. Graphpromptliu2023graphpromptis developed to unify graph prompt templates and enhance their performance through learnable readout prompt functions\. All\-in\-Onesun2023allproposed a novel graph token structure accompanied to aligning pre\-training and fine\-tuning objectives in the context of graph\-based tasks\. GPFfang2024universalintegrates the feature prompt in a more general way to improve the prompt performance\. HGPromptyu2024hgpromptdesigns meta\-path\-based prompts for heterogeneous graphs\. SUPTlee2024subgraphproposes subgraph\-level universal graph prompt learning, while LEAPxu2025learningintroduces a reinforcement learning \(RL\) based approach for editing universal graph prompts in molecular graphs\. However, leveraging prompt\-based techniques for crystal property prediction remains largely unexplored by the research community\. Also, current graph prompt\-tuning methods operate within the pre\-train/fine\-tune paradigm, and to the best of our knowledge, no prior work has explored prompt\-tuning for supervised learning from scratch\.

## Appendix DAdditional Experimental Results

GNNModelSetup\# ParametersFormationEnergyBandgap\(OPT\)TotalEnergyEhullBandgap\(MBJ\)Bulk Modulus\(Kv\)Shear Modulus\(Gv\)SLME\(%\)SpillageVanilla0\.423M0\.0630\.2000\.0780\.1700\.41014\.4711\.755\.660\.3972×\\timesGNN Layers0\.788M0\.0610\.1740\.0730\.1620\.42914\.1511\.395\.600\.402Increase Hidden Dim1\.435M0\.0550\.1770\.0720\.1750\.41914\.7411\.595\.310\.416CGCNNSkip Connection0\.423M0\.0590\.1920\.0730\.1610\.40415\.3712\.305\.660\.429Adapter Learning0\.476M0\.1510\.2240\.2780\.2610\.51214\.6211\.364\.980\.410Prompt Learning0\.433M0\.0410\.1550\.0470\.1160\.35412\.5410\.584\.810\.388Vanilla3\.27M0\.0330\.1370\.0350\.0640\.30011\.2110\.765\.260\.3982×\\timesGNN Layers6\.34M0\.0320\.1430\.0350\.0570\.31610\.429\.605\.110\.367Increase Hidden Dim12\.84M0\.0520\.1390\.0920\.1040\.32810\.859\.934\.760\.367MatformerSkip Connection3\.27M0\.0330\.1380\.0340\.0640\.31310\.8110\.784\.870\.368Adapter Learning3\.36M0\.0890\.1560\.1940\.1550\.36013\.2811\.025\.150\.380Prompt Learning3\.28M0\.0320\.1360\.0330\.0530\.28510\.059\.574\.760\.360

Table 6:Summary of results \(MAE\) for various properties in JARVIS\-DFT using different architectural modifications in CGCNN and Matformer\. Our proposed prompt learning paradigm is both lightweight and delivers superior performance compared to alternative approaches\.EncoderStrategyTuningStrategyFormationEnergyBandgap\(OPT\)TotalEnergyEhullBandgap\(MBJ\)BulkModulus\(Kv\)ShearModulus\(Gv\)SLME\(%\)SpillageCGCNNVanilla0\.0630\.2000\.0780\.1700\.41014\.4711\.755\.660\.397Only Node0\.0410\.1600\.0480\.1170\.34412\.3510\.384\.760\.384Only Graph0\.0420\.1560\.0490\.1190\.34112\.2510\.514\.920\.391Both0\.0410\.1550\.0470\.1160\.35412\.5410\.584\.800\.388ALIGNNVanilla0\.0350\.1420\.0370\.0760\.31010\.409\.864\.520\.351Only Node0\.0340\.1410\.0350\.0660\.29210\.249\.714\.490\.346Only Graph0\.0340\.1410\.0360\.0650\.29110\.389\.624\.470\.350Both0\.0330\.1400\.0350\.0640\.29910\.219\.734\.420\.348MatformerVanilla0\.0330\.1370\.0350\.0640\.30011\.2110\.765\.260\.398Only Node0\.0320\.1360\.0340\.0560\.29510\.219\.534\.770\.359Only Graph0\.0330\.1380\.0340\.0560\.29710\.269\.624\.650\.355Both0\.0320\.1360\.0330\.0530\.28510\.059\.574\.760\.360PotNetVanilla0\.02940\.1270\.03210\.0550\.27010\.119\.234\.570\.361Only Node0\.02940\.1270\.03170\.0530\.2519\.958\.804\.380\.349Only Graph0\.02990\.1290\.03140\.0490\.2669\.648\.984\.220\.350Both0\.02920\.1270\.03100\.0480\.2589\.989\.014\.430\.352

Table 7:Ablation results with different modalities of prompt learning strategies\. Best results isboldand the second best result isunderlined\.### D\.1Alternate Architectural Modifications

Next, we explore alternative methods for learning latent chemical and structural features by making simple modifications to the existing GNN models\. The approaches investigated are increasing hidden layer dimensions, doubling the number of GNN layers, introducing skip connections, and implementing adapter learning\.

1. 1\.2×\\timesGNN Layers\.In this variant, the number of GNN layers is increased by doubling the original number of layers\.
2. 2\.Increasing Hidden Dim\.In this variant, the hidden layer dimensions for each GNN are increased by 64\.
3. 3\.Skip Connection\.In this variant, residual connections are added to each layer of the GNN\.
4. 4\.Adapter Learning\.In this variant, a parameter\-efficient adapter\-based tuning technique is used for each layer of the GNN\. The method follows the approach proposed in AdapterGNNli2024adaptergnn, where parameter\-efficient adapters are added to each layer of the GNN models\.

We conduct another set of experiments using JARVIS\-DFT dataset on CGCNN and Matformer to compare the performance of these alternative methods with prompt learning and report the results in Table[6](https://arxiv.org/html/2607.08996#A4.T6)\. We observe that compared to all other approaches, prompt learning introduces minimal parameter overhead while outperforming all of these\. In particular, increasing hidden layer dimensions or doubling the number of GNN layers substantially increases the model’s parameter count, yet results in minimal performance improvement across all properties\. On the other hand, while adapter learning and skip connections add only a small parameter overhead, they also fail to significantly enhance crystal property prediction performance\. This highlights the ingenuity of our graph prompt learning approach, which is lightweight, model\-agnostic, and provides substantial improvement over the vanilla models\.

### D\.2Impact of Different Prompt Modalities\.

In the main paper, a brief summary of this result is provided; here, we present an elaboration on some of the details\. We conduct an ablation study to investigate the influence of different prompt modalities in enhancing SOTA model performance and report the results in Table[7](https://arxiv.org/html/2607.08996#A4.T7)\. In specific, we conduct experiments with three setups\. 1\) By default, in our proposed framework we infused both node and graph\-level prompts \(denoted as*"Both"*\) 2\) With only node\-level prompts infused at input node\-features and no graph\-level prompts \(denoted as*"Only Node"*\) 3\) With only graph\-level prompts infused at graph embedding and no node\-level prompts \(denoted as*"Only Graph"*\) For this study, we used four baseline models: CGCNN, ALIGNN, Matformer and PotNet, each trained on nine properties from the JARVIS\-DFT dataset: formation energy, bandgap \(OPT\), total energy, Ehull, bandgap\(MBJ\), bulk modulus\(Kv\), shear modulus\(Gv\), SLME\(%\), and Spillage\. From Table[7](https://arxiv.org/html/2607.08996#A4.T7)it is evident that using both node\-level and graph\-level prompt learning together \(Both\), CGCNN, ALIGNN, Matformer, and PotNet yields the best results in most of the cases \(15 out of 36 experiments\), surpassing the performance of using either node\-level or graph\-level prompt learning alone\.

PropertyCGCNNCGCNN\(Prompt\)ALIGNNALIGNN\(Prompt\)Mean±\\pmStdCIP\-ValueMean±\\pmStdCIP\-ValueFormation Energy0\.0630\.041±\\pm0\.010\(0\.0286, 0\.0534\)0\.00790\.0350\.033±\\pm0\.001\(0\.0318, 0\.0342\)0\.0110Bandgap0\.2000\.155±\\pm0\.020\(0\.1302, 0\.1798\)0\.00730\.1420\.140±\\pm0\.001\(0\.1388, 0\.1412\)0\.0110Total Energy0\.0780\.047±\\pm0\.010\(0\.0346, 0\.0594\)0\.00230\.0370\.035±\\pm0\.0009\(0\.0339, 0\.0361\)0\.0077Ehull0\.1700\.116±\\pm0\.020\(0\.0912, 0\.1408\)0\.00380\.0760\.064±\\pm0\.004\(0\.0590, 0\.0690\)0\.0026mbj Bandgap0\.4100\.354±\\pm0\.010\(0\.3416, 0\.3664\)0\.00020\.3100\.299±\\pm0\.003\(0\.2953, 0\.3027\)0\.0012Bulk Modulus14\.47012\.540±\\pm1\.100\(11\.1742, 13\.9058\)0\.017210\.40010\.210±\\pm0\.110\(10\.0734, 10\.3466\)0\.0181Shear Modulus11\.75010\.580±\\pm0\.300\(10\.2075, 10\.9525\)0\.00109\.8629\.731±\\pm0\.090\(9\.6193, 9\.8427\)0\.0312SLME5\.6604\.801±\\pm0\.400\(4\.3043, 5\.2977\)0\.00864\.5214\.420±\\pm0\.060\(4\.3455, 4\.4945\)0\.0197Spillage0\.3970\.388±\\pm0\.005\(0\.3818, 0\.3942\)0\.01580\.3510\.348±\\pm0\.002\(0\.3455, 0\.3505\)0\.0285

\(a\)
PropertyMatformerMatformer\(Prompt\)PotNetPotNet\(Prompt\)Mean±\\pmStdCIP\-ValueMean±\\pmStdCIP\-ValueFormation Energy0\.0330\.032±\\pm0\.0007\(0\.0311, 0\.0329\)0\.0330\.02940\.0292±\\pm0\.0001\(0\.0291, 0\.0293\)0\.011Bandgap0\.1370\.136±\\pm0\.0005\(0\.1354, 0\.1366\)0\.0110\.12740\.1270±\\pm0\.0002\(0\.1268, 0\.1272\)0\.011Total Energy0\.0350\.033±\\pm0\.001\(0\.0318, 0\.0342\)0\.0110\.03240\.0311±\\pm0\.001\(0\.0299, 0\.0323\)0\.044Ehull0\.0640\.053±\\pm0\.004\(0\.0480, 0\.0580\)0\.00350\.05520\.0481±\\pm0\.004\(0\.0431, 0\.0531\)0\.0166mbj Bandgap0\.3000\.285±\\pm0\.010\(0\.2726, 0\.2974\)0\.0280\.27010\.258±\\pm0\.007\(0\.2493, 0\.2667\)0\.018Bulk Modulus11\.2110\.05±\\pm0\.720\(9\.1560, 10\.9440\)0\.022710\.119\.981±\\pm0\.090\(9\.8693, 10\.0927\)0\.0327Shear Modulus10\.769\.581±\\pm0\.511\(8\.9465, 10\.2155\)0\.00679\.2329\.0201±\\pm0\.102\(8\.8935, 9\.1467\)0\.0097SLME5\.264\.764±\\pm0\.322\(4\.3642, 5\.163\)0\.0264\.57014\.431±\\pm0\.104\(4\.3019, 4\.5601\)0\.040Spillage0\.3980\.359±\\pm0\.020\(0\.3342, 0\.3838\)0\.0120\.3610\.3522±\\pm0\.003\(0\.3485, 0\.3559\)0\.0028

\(b\)
PropertyiComFormeriComFormer\(Prompt\)eComFormereComFormer\(Prompt\)Mean±\\pmStdCIP\-ValueMean±\\pmStdCIP\-ValueFormation Energy0\.0270\.026±\\pm0\.0003\(0\.0256, 0\.0264\)0\.00170\.0280\.027±\\pm0\.0003\(0\.0266, 0\.0274\)0\.0017Bandgap0\.1350\.134±\\pm0\.0004\(0\.1335, 0\.1345\)0\.00500\.1240\.122±\\pm0\.001\(0\.1208, 0\.1232\)0\.0111Total Energy0\.0310\.029±\\pm0\.001\(0\.0279, 0\.0301\)0\.00770\.0320\.031±\\pm0\.0002\(0\.0308, 0\.0312\)0\.00036Ehull0\.0470\.046±\\pm0\.0002\(0\.0458, 0\.0462\)0\.000360\.0440\.040±\\pm0\.001\(0\.0388, 0\.0412\)0\.00086mbj Bandgap0\.2640\.259±\\pm0\.003\(0\.2553, 0\.2627\)0\.02040\.2800\.276±\\pm0\.002\(0\.2735, 0\.2785\)0\.0111Bulk Modulus14\.2713\.22±\\pm0\.72\(12\.3260, 14\.1140\)0\.031113\.4512\.26±\\pm0\.825\(11\.2356, 13\.2844\)0\.0321Shear Modulus9\.769\.359±\\pm0\.201\(9\.1094, 9\.6086\)0\.011211\.1210\.56±\\pm0\.32\(10\.1627, 10\.9573\)0\.0173SLME4\.7024\.342±\\pm0\.23\(4\.0564, 4\.6276\)0\.02494\.6624\.650±\\pm0\.004\(4\.6450, 4\.6550\)0\.0026Spillage0\.3690\.351±\\pm0\.01\(0\.3386, 0\.3634\)0\.01580\.3820\.369±\\pm0\.009\(0\.3578, 0\.3802\)0\.0320

\(c\)

Table 8:Statistical comparison ofpromptversion across six backbone models\.
### D\.3Statistical Significance of The Results

We perform a comprehensive statistical analysis to ensure the robustness and reliability of the reported performance improvements\. For each prompt\-enhanced variant, we conduct five independent runs with different random seeds and report the mean, standard deviation, and 95% confidence interval \(CI\)\. In addition, we compute paired t\-test p\-values for each backbone to assess the statistical significance of the observed improvements\. Specifically, we evaluate CGCNN, ALIGNN, Matformer, PotNet, iComFormer, and eComFormer, along with their corresponding prompt\-enhanced variants that use these encoders as backbones\. The complete results on the JARVIS dataset are presented in Table[8](https://arxiv.org/html/2607.08996#A4.T8)\. These statistical measures demonstrate that the performance gains achieved by the prompt\-enhanced models are consistent and reproducible across multiple runs\. Notably, the paired t\-tests yield p\-values below 0\.05 for most evaluated properties, indicating statistically significant improvements\. Overall, this analysis confirms that the improvements obtained through prompt learning do not arise from random variation but instead reflect genuine and meaningful performance gains across downstream tasks\.

BackboneMethodParams \(M\)Total TrainingTime/Epoch \(s\)Total TrainingFLOPs\(G\)Time\(s\)CGCNNGPF0\.323M2\.53×106\\times 10^\{6\}14\.6314,630SUPT0\.326M2\.90×106\\times 10^\{6\}29\.3329,334LEAP0\.954M1\.65×108\\times 10^\{8\}12501,250,000Prompt \(Ours\)0\.433M2\.54×106\\times 10^\{6\}14\.7414,740MatformerGPF2\.18M2\.10×106\\times 10^\{6\}8585,000SUPT11\.5M2\.80×106\\times 10^\{6\}110110,000LEAP19\.4M1\.80×108\\times 10^\{8\}4043\.44,043,400Prompt \(Ours\)3\.28M2\.27×106\\times 10^\{6\}9090,000

Table 9:Efficiency comparison of graph prompt learning methods on crystal property prediction tasks\.ModelVariantParamsExtra ParamsTrain Time \(s/epoch\)GPU \(GB\)Infer Time \(ms\)MatformerVanilla3\.27M—142\.89\.136\.461Prompt3\.28M0\.01M143\.29\.166\.510PotNetVanilla0\.473M—47\.05\.823\.060Prompt0\.484M0\.011M47\.25\.953\.120iComFormerVanilla1\.38M—72\.39\.514\.200Prompt1\.39M0\.01M72\.69\.624\.250eComFormerVanilla3\.72M—77\.815\.904\.580Prompt3\.73M0\.01M78\.016\.104\.600

Table 10:Efficiency comparison between vanilla and prompt\-infused variants across four backbone GNN models\. Prompt learning introduces negligible overhead in parameters, training time, GPU memory, and inference latency\.
### D\.4Efficiency Study

We analyze the computational efficiency of different prompt\-tuning strategies in terms of the number of trainable parameters, total training FLOPs \(measured in GFLOPs\), time per epoch, and total training time, as reported in Table[9](https://arxiv.org/html/2607.08996#A4.T9)\. Compared to SUPT and LEAP, our prompt\-based variants of CGCNN and Matformer require substantially fewer training FLOPs while maintaining performance comparable to GPF\. In particular, LEAP incurs significantly higher computational cost due to its reinforcement learning–based prompt optimization, resulting in orders\-of\-magnitude larger training FLOPs and training time\. In contrast, our prompt\-infused approach introduces only a small number of additional parameters and preserves the lightweight nature of the backbone models, leading to efficient training with minimal computational overhead\. To further quantify this overhead relative to the underlying backbones themselves, Table[10](https://arxiv.org/html/2607.08996#A4.T10)reports a direct comparison between the vanilla and prompt\-infused variants of four GNN backbones: Matformer, PotNet, iComFormer, and eComFormer – in terms of parameter count, training time per epoch, peak GPU memory usage, and inference time\. The proposed prompts introduce only a negligible number of additional parameters \(≈\\approx0\.01M across all backbones; e\.g\., Matformer: 3\.27M→\\rightarrow3\.28M\), under 0\.5% for larger backbones such as Matformer and eComFormer, and at most 2\.3% for the smallest \(PotNet\)\. This overhead is similarly minimal in practice: training time per epoch increases only marginally \(typically 0\.3–0\.7% across backbones\), peak GPU memory usage remains nearly unchanged \(0–2%\), and inference time shows negligible overhead \(0\.4–2% across backbones\)\. This efficiency stems from the design of the prompt modules, which consist of lightweight linear layers and embedding operations rather than costly additions such as extra message\-passing or attention layers, allowing the framework to remain model\-agnostic without sacrificing the computational efficiency of the underlying backbone\. Overall, the proposed method achieves consistent performance improvements with minimal computational and memory overhead, making it practical for real\-world deployment\. All experiments were conducted on the JARVIS\-DFT dataset using a single NVIDIA A6000 GPU to ensure a fair comparison across methods\. This favorable trade\-off also holds when viewed analytically rather than just empirically\. Withkknode prompt bases and seven crystal\-system graph\-level prompts, the total additional parameters amount toO​\(\(k\+7\)​d\)O\(\(k\+7\)d\), a modest overhead relative to the multi\-layer GNN backbone\. Consequently, the proposed method enhances model expressiveness without meaningfully increasing the risk of overfitting\.

NoiseSLME \(%\)Shear Modulus \(Gv\)Bulk Modulus \(Kv\)Bandgap \(MBJ\)0\.004\.7649\.58110\.0500\.2850\.014\.7849\.59710\.1200\.2890\.054\.8019\.62110\.1400\.2910\.104\.8129\.65410\.1900\.295

Table 11:Sensitivity of graph\-level prompts to noise added to the crystal system label\. Noise denotes the label corruption probability\.
ModelSettingSLME \(%\)Shear Modulus \(Gv\)Bulk Modulus \(Kv\)Bandgap \(MBJ\)Matformer \(Only Graph\)OracleSG \(Ours\)4\.659\.6210\.260\.297UnknownSG4\.999\.8610\.820\.302Matformer \(Both\)OracleSG \(Ours\)4\.769\.5710\.050\.285UnknownSG4\.969\.8810\.770\.291Unsupervised4\.939\.8310\.790\.321

Table 12:Effect of space group \(SG\) availability on graph\-level prompting for Matformer\.

### D\.5Robustness to Label Noise and Generalization under Distorted, Label\-Free Crystal Inputs

We evaluate robustness under increasing noise levels and in the absence of ground\-truth labels, using the Matformer backbone for all experiments in this study\. As shown in Table[12](https://arxiv.org/html/2607.08996#A4.T12), performance degrades only marginally as the noise level increases from 0 to 0\.1 \(e\.g\., SLME: 4\.764→\\rightarrow4\.812\), indicating that the model remains stable under input perturbations\. To assess label dependence, we further compare against a setting where crystal system labels are unavailable \(UnknownSG\), which leads to moderate degradation relative to the OracleSG setting \(Table[12](https://arxiv.org/html/2607.08996#A4.T12)\)\. Introducing a clustering\-based unsupervised prompt in place of ground\-truth labels recovers most of this gap, achieving performance close to the oracle setting\. Together, these results suggest that graph\-level prompts are robust to both input distortions and missing supervision, and can be effectively learned in a fully unsupervised manner directly from structural information\.

PropertyGPFSUPTLEAPPrompt \(Ours\)Formation Energy0\.0320\.0310\.0390\.027Bandgap \(OPT\)0\.1350\.1390\.1340\.120Total Energy0\.0310\.0310\.0450\.029Ehull0\.0610\.0580\.0780\.054Bandgap \(MBJ\)0\.3120\.2920\.3720\.287Bulk Modulus10\.6910\.6612\.1310\.30Shear Modulus9\.6599\.55810\.359\.498SLME4\.8224\.9094\.7894\.724Spillage0\.3620\.3590\.3830\.347

Table 13:Comparison of graph prompt\-tuning baselines \(GPF, SUPT, LEAP\) against our proposed prompt learning framework on nine properties from the JARVIS\-DFT dataset\. Best result in each row is highlighted inbold\.ModelSettingsSLME \(%\)Shear Modulus \(Gv\)Bulk Modulus \(Kv\)Bandgap \(MBJ\)Matformernode\-prompt \+ Crystal System OneHot4\.8309\.67010\.7000\.296node\-prompt \+ Crystal System Embedding4\.8249\.61010\.1500\.291Conditional Bias4\.8519\.96110\.8450\.301Prompt \(Ours\)4\.7609\.58110\.0500\.285

Table 14:Comparison of graph\-level conditioning strategies for Matformer on JARVIS\-DFT\. Best result in each column is highlighted inbold\.
### D\.6Disentangling the Contribution of Graph\-Level Prompts

To further isolate the contribution of graph\-level prompts, we compare our method against several baselines that incorporate crystal system information through simpler mechanisms: one\-hot concatenation, learned embedding concatenation, and conditional bias \(feature\-wise affine conditioning via scaling and shifting\)dumoulin2018feature\-wise\. As shown in Table[14](https://arxiv.org/html/2607.08996#A4.T14), access to crystal system information yields only modest gains, and the extent of improvement depends strongly on the integration mechanism\. Embedding\-based concatenation slightly outperforms one\-hot encoding, while the conditional bias baseline fails to improve performance, consistently yielding higher MAE across all evaluated properties\. These results suggest that the observed gains are not simply a consequence of exposing the model to crystal system information, but instead depend critically on how that information is integrated, supporting the effectiveness of our proposed graph\-level prompting mechanism\.

### D\.7Fairness of Baseline Comparison

In the main paper, baseline results for graph prompt\-tuning methods \(GPF, SUPT, LEAP\) are reported using the settings and hyperparameters published in their original papers\. This is standard practice in the graph prompt\-tuning literature, since each baseline’s hyperparameters, training schedule, and implementation\-specific details have already been optimized and validated by its original authors through peer review\. Forcing all baselines through a single, uniform pipeline can instead introduce its own sources of unfairness, potentially degrading a baseline’s performance relative to what its authors demonstrated and misrepresenting its true capability\. To further validate our comparisons, we additionally conduct two controlled experiments that place all baselines under matched training conditions\. First, we re\-implement GPF, SUPT, and LEAP from scratch on a unified backbone \(Matformer\), training all methods under identical protocols \(optimizer, learning rate schedule, number of epochs, and data splits\)\. Second, to further isolate the effect of pretraining, we fine\-tune all methods on top of the same DPF pre\-trained Matformer checkpoint, again under identical fine\-tuning settings\. As shown in Table[13](https://arxiv.org/html/2607.08996#A4.T13), unifying the training protocol modestly improves GPF, SUPT, and LEAP relative to their originally reported numbers, but our method continues to consistently outperform all three across every evaluated property in both the from\-scratch and DPF fine\-tuning settings\. This indicates that our gains stem from the proposed prompt design itself, rather than from differences in backbone, training strategy, or pretraining\. We attribute the remaining gap to these baselines reliance on node\-centric prompting mechanisms: GPF operates at the node\-level, SUPT at the subgraph level, and LEAP performs reinforcement\-learning\-based prompt editing – none of which explicitly model crystal\-level structural symmetry\. In contrast, CrysPrompt incorporates both node\-level and crystal\-system\-aware graph\-level prompts, jointly capturing local chemical semantics and global structural symmetry\. This multi\-level design yields consistent improvements even under matched training conditions, with average gains of 7\.12% over GPF, 5\.85% over SUPT, and 18\.27% over LEAP on JARVIS\-DFT, demonstrating the effectiveness of our proposed multi\-level prompt design\.

![Refer to caption](https://arxiv.org/html/2607.08996v1/x5.png)Figure 5:Periodic table visualization of the mean node\-level prompt embedding values for dimension 84 across elements\. Tile colors denote the prompt embedding values, while the numbers inside each tile represent the electronegativity of the corresponding element \(Pauling scale\)\.![Refer to caption](https://arxiv.org/html/2607.08996v1/x6.png)Figure 6:Periodic table visualization of the mean node\-level prompt embedding values for dimension 73 across elements\. Tile colors represent prompt values, while the numbers denote the first ionization energy \(eV\) of each element\. The learned prompt dimension exhibits periodic trends consistent with the variation of ionization energy across the periodic table\.
### D\.8Expressiveness of Node\-Level Prompt

To further analyze the expressiveness of the learned node\-level prompts, we investigate whether individual prompt dimensions capture meaningful chemical trends across elements\. For each element, we compute the mean value of the learned prompt embeddings across all occurrences of that element in the dataset\. These aggregated values are then visualized over the periodic table to examine whether the learned prompt representations exhibit patterns consistent with known chemical properties\. Figs\.[5](https://arxiv.org/html/2607.08996#A4.F5)and[6](https://arxiv.org/html/2607.08996#A4.F6)illustrate representative prompt dimensions that correlate with fundamental elemental descriptors\. In Figure 5, the tile colors represent the mean prompt embedding values for dimension 84, while the numbers inside each tile denote the Pauling electronegativity of the corresponding element\. The learned prompt values follow periodic patterns consistent with electronegativity variations across the periodic table\. Elements with higher electronegativity—such as O, F, Cl, and Br—tend to exhibit larger prompt values, while elements with lower electronegativity, particularly heavier electropositive metals \(e\.g\., Cs and Ba\), generally correspond to comparatively smaller prompt valuespauling1932nature\. Quantitatively, we observe a strong Pearson correlation of 0\.784 between the prompt values and electronegativity\. As illustrated in Fig\.[6](https://arxiv.org/html/2607.08996#A4.F6), prompt dimension 73 exhibits periodic patterns consistent with the variation of first ionization energy across the periodic table\. Electropositive metals with low ionization energies, such as Cs, Rb, and K, tend to correspond to smaller prompt values \(lighter colors\)\. In contrast, transition metals with comparatively higher ionization energies, such as Pt, Au, and Hg, exhibit larger prompt values \(darker colors\)mulliken1934new\. Quantitatively, we observe a Pearson correlation coefficient ofr=r=0\.742 between the prompt values and the ionization energy, indicating that the learned prompt dimension captures chemically significant trends related to the atomic electron binding strength\.

![Refer to caption](https://arxiv.org/html/2607.08996v1/x7.png)Figure 7:Visualization of node\-level prompt embeddings across periodic blocks\. Two\-dimensional t\-SNE projection of node\-level prompt embeddings\. Each point represents a chemical element and is colored by its periodic block \(s, p, d, or f\)\.![Refer to caption](https://arxiv.org/html/2607.08996v1/x8.png)Figure 8:Visualization of graph prompt embeddings across crystal systems\. Two\-dimensional projection of graph\-level prompt embeddings using principal component analysis \(PCA\)\. Each point represents a crystal structure and is colored according to its crystal systems\.
### D\.9Expressiveness of Graph\-Level Prompt

To evaluate whether the learned graph\-level prompts capture meaningful structural information, we analyze the prompt embeddings using unsupervised clustering\. Specifically, we apply K\-means clustering withk=k=7, corresponding to the seven crystal systems, on the learned graph prompt embeddings and visualize them using Principal Component Analysisabdi2010principal\. As shown in Fig\.[8](https://arxiv.org/html/2607.08996#A4.F8), the embeddings form well\-separated clusters that align with crystallographic symmetry groups\. Quantitatively, the clustering achieves Normalized Mutual Information \(NMI\)=0\.70, Adjusted Rand Index \(ARI\)=0\.75, and a Silhouette score of 0\.642, indicating that the learned prompt embeddings capture significant structural information about crystal symmetry\. The relatively high agreement between unsupervised clusters and ground\-truth crystal system labels suggests that the graph\-level prompts encode meaningful global geometric characteristics of crystalline materials\. Overall, this analysis demonstrates that the learned prompts are expressive and capture structural patterns consistent with crystallographic symmetry\.