DPTM-DT: 双预训练Transformer多任务表示学习用于药物-靶点预测

# DPTM-DT: Dual-Pretrained Transformer Multitask Representation Learning for Drug-Target Prediction
Source: [https://arxiv.org/html/2609.22184](https://arxiv.org/html/2609.22184)
Ge KongAddress:School of Biological Science and Medical Engineering, Beihang University, Beijing 100191, China E\-mail:[gekong@buaa\.edu\.cn](mailto:[email protected])

###### Abstract

Drug\-target relation prediction supports candidate screening, drug repositioning, and mechanism analysis\. Existing models often use incomplete drug or protein representations, model cross\-modal interactions shallowly, or train affinity regression and interaction classification separately, although these tasks describe closely related views of the same drug\-target pair\. This paper presents DPTM\-DT, a dual\-pretrained Transformer framework for multitask drug\-target prediction\. DPTM\-DT combines GROVER molecular graph embeddings, ESM protein language\-model embeddings, and CTD physicochemical descriptors, then exchanges drug\-target information through bidirectional cross\-modal attention\. A shared pair representation is used for continuous affinity regression, high\-affinity binary classification, and six\-level affinity classification\. Experiments on Davis and KIBA cover random 80/20 and DeepDTA\-style standard splits\. On the random 80/20 split, DPTM\-DT achieves MSE/CI values of 0\.193/0\.917 on Davis and 0\.120/0\.918 on KIBA\. It also reports binary AUPR/MCC values of 0\.727/0\.654 and 0\.798/0\.689, and six\-class Macro\-F1/Top\-2 values of 0\.800/0\.932 and 0\.815/0\.962 on Davis and KIBA, respectively\. Across the reported regression, binary classification, and multiclass classification settings, DPTM\-DT achieves the best overall performance among the compared methods\. Results under the standard split show the same relative trend\. Ablations indicate that dual target representation, gated fusion, and cross\-modal attention each contribute to the final performance\. Code and supplementary materials are available at:[anonymous\.4open\.science/r/DPCM\-DT\-74E0](https://anonymous.4open.science/r/DPCM-DT-74E0)\.

###### Keywords:

drug\-target prediction , binding affinity prediction , cross\-modal learning , multitask learning , pretrained representation

## 1Introduction

Drug\-target interaction \(DTI\) and drug\-target affinity \(DTA\) prediction are central tasks in computational drug discovery\. Experimental validation is reliable but expensive, while the number of possible compound\-protein pairs is large\. Computational models can therefore help prioritize candidates before wet\-lab testing\. Recent deep methods learn from SMILES strings, molecular graphs, and protein sequences, improving on earlier descriptor\- and similarity\-based approaches\[[1](https://arxiv.org/html/2609.22184#bib.bib1),[2](https://arxiv.org/html/2609.22184#bib.bib2),[3](https://arxiv.org/html/2609.22184#bib.bib3),[4](https://arxiv.org/html/2609.22184#bib.bib4),[5](https://arxiv.org/html/2609.22184#bib.bib5),[6](https://arxiv.org/html/2609.22184#bib.bib6)\]\.

Existing DTI and DTA models still face three limitations\. First, a single representation rarely captures both molecular topology and protein sequence or physicochemical information\. Second, simple concatenation cannot fully describe the correspondence between drug and target modalities\. Third, regression, binary classification, and affinity\-level classification are related views of the same pairwise relation but are often optimized separately\. DPTM\-DT addresses these limitations through dual pretrained representations, cross\-modal Transformer interaction, and multitask prediction\.

DPTM\-DT addresses these issues with a multitask architecture in which representation learning, cross\-modal interaction, and task supervision are trained around the same drug\-target pair\. The design has three parts: dual pretrained drug and protein representations, bidirectional drug\-target information exchange, and a shared pair representation for multiple affinity\-oriented objectives\. The paper makes three contributions: \(1\) a dual\-pretrained representation module that combines GROVER, ESM, and CTD features; \(2\) a cross\-modal joint interaction module for pairwise drug\-target matching; and \(3\) an evaluation protocol covering affinity regression, binary interaction classification, and six\-class affinity\-level classification on Davis and KIBA under two split protocols\.

## 2Related Work

Drug\-target prediction has moved from handcrafted descriptors and similarity kernels toward end\-to\-end representation learning\. DeepDTA encodes SMILES strings and protein sequences with convolutional networks, showing that affinity prediction can be learned directly from raw symbolic inputs\[[3](https://arxiv.org/html/2609.22184#bib.bib3)\]\. WideDTA extends this idea with word\-level drug and protein representations, including motifs, domains, and molecular substructures\[[4](https://arxiv.org/html/2609.22184#bib.bib4)\]\. Graph\-based models such as GLCN\-DTA further exploit molecular topology, while TEFDTA combines Transformer encoding with molecular fingerprints for affinity prediction\[[5](https://arxiv.org/html/2609.22184#bib.bib5),[6](https://arxiv.org/html/2609.22184#bib.bib6)\]\. Complementary feature\-fusion DTA models have also explored multi\-level sequence features, hybrid fingerprints with protein N\-grams, and E3FP\-based multimodal molecular representations\[[7](https://arxiv.org/html/2609.22184#bib.bib7),[8](https://arxiv.org/html/2609.22184#bib.bib8),[9](https://arxiv.org/html/2609.22184#bib.bib9)\]\. Recent methods continue this trend through multimodal fusion, dual attention, cross\-scale graph contrastive learning, and multi\-task co\-attention\[[10](https://arxiv.org/html/2609.22184#bib.bib10),[11](https://arxiv.org/html/2609.22184#bib.bib11),[12](https://arxiv.org/html/2609.22184#bib.bib12),[13](https://arxiv.org/html/2609.22184#bib.bib13),[14](https://arxiv.org/html/2609.22184#bib.bib14),[15](https://arxiv.org/html/2609.22184#bib.bib15),[16](https://arxiv.org/html/2609.22184#bib.bib16),[17](https://arxiv.org/html/2609.22184#bib.bib17)\]\. Reviews of graph neural networks and Transformer models in drug discovery also suggest that representation quality and evaluation protocol design remain central issues for DTI and DTA models\[[2](https://arxiv.org/html/2609.22184#bib.bib2),[18](https://arxiv.org/html/2609.22184#bib.bib18)\]\. DPTM\-DT follows these representation\-learning models while emphasizing protein\-side complementarity and explicit drug\-target interaction\.

Recent pretrained models provide a natural way to strengthen each modality before drug\-target matching\. GROVER learns molecular graph semantics from large\-scale unlabeled molecules, and ESM learns contextual protein sequence representations from evolutionary\-scale protein data\[[19](https://arxiv.org/html/2609.22184#bib.bib19),[20](https://arxiv.org/html/2609.22184#bib.bib20)\]\. In parallel, CTD descriptors remain useful because they summarize physicochemical protein properties in a compact and interpretable form\[[31](https://arxiv.org/html/2609.22184#bib.bib31)\]\. More broadly, foundation models and molecular representation learning have become increasingly important for drug discovery because they can transfer reusable structural and sequence information across downstream tasks\[[21](https://arxiv.org/html/2609.22184#bib.bib21),[22](https://arxiv.org/html/2609.22184#bib.bib22)\]\. Recent studies have also applied graph attention and pretrained language models to biomedical omics, protein localization, and RNA localization prediction tasks\[[23](https://arxiv.org/html/2609.22184#bib.bib23),[24](https://arxiv.org/html/2609.22184#bib.bib24),[25](https://arxiv.org/html/2609.22184#bib.bib25)\]\. These studies indicate that the field is moving toward richer molecular and protein representations, but many evaluations still emphasize a single prediction target or a single split protocol\. DPTM\-DT follows this direction by combining pretrained molecular and protein embeddings with CTD features, then using cross\-modal attention and multitask heads to evaluate the same pair representation under regression, binary classification, and affinity\-level classification\.

## 3Method

### 3\.1Overall Pipeline

DPTM\-DT models a sample as a drug\-target pair\(di,pi\)\(d\_\{i\},p\_\{i\}\)with a continuous affinity labelaia\_\{i\}\. As shown in Fig\.[1](https://arxiv.org/html/2609.22184#S3.F1), the model first extracts drug\-side and target\-side representations, maps all modalities into a shared 256\-dimensional space, performs target\-side feature fusion, updates the pair through cross\-modal interaction, and finally predicts three task outputs\. The three supervised tasks reuse the same pair representation: continuous affinity regression, binary interaction classification, and six\-class affinity\-level prediction\.

![Refer to caption](https://arxiv.org/html/2609.22184v1/1.png)Figure 1:Overall architecture of DPTM\-DT\. The model encodes drug SMILES with GROVER, encodes target FASTA sequences with ESM\-2 and CTD descriptors, fuses target\-side features, performs bidirectional cross\-modal Transformer interaction, and predicts affinity regression, binary interaction, and six\-level affinity classes from the shared pair representation\.
### 3\.2Drug and Target Representation

On the drug side, DPTM\-DT uses GROVER to encode molecular graph information\. The resulting drug vector is denoted asfid​r​u​g∈ℝ4800f\_\{i\}^\{drug\}\\in\\mathbb\{R\}^\{4800\}\. This representation captures molecular topology, local atom\-bond patterns, and graph\-level chemical semantics learned from large\-scale molecular pretraining\.

On the target side, the model uses two complementary sources\. ESM provides a protein language\-model embeddingfie​s​m∈ℝ960f\_\{i\}^\{esm\}\\in\\mathbb\{R\}^\{960\}, encoding sequence context and long\-range dependencies\. CTD provides a 147\-dimensional descriptorfic​t​d∈ℝ147f\_\{i\}^\{ctd\}\\in\\mathbb\{R\}^\{147\}based on composition, transition, and distribution statistics over physicochemical properties\[[31](https://arxiv.org/html/2609.22184#bib.bib31)\]\. These properties include hydrophobicity, polarity, charge, van der Waals volume, polarizability, solvent accessibility, and secondary\-structure tendency\.

All input modalities are projected into a common hidden space:

vim=Πm​\(fim\),m∈\{d​r​u​g,e​s​m,c​t​d\},v\_\{i\}^\{m\}=\\Pi\_\{m\}\(f\_\{i\}^\{m\}\),\\quad m\\in\\\{drug,esm,ctd\\\},\(1\)whereΠm\\Pi\_\{m\}is a modality\-specific projection with nonlinearity, dropout, and layer normalization\. The target branch then fuses ESM and CTD\. The full model uses gated fusion:

gi=σ⁡\(Wg​\[vie​s​m;vic​t​d\]\+bg\),g\_\{i\}=\\sigma\(W\_\{g\}\[v\_\{i\}^\{esm\};v\_\{i\}^\{ctd\}\]\+b\_\{g\}\),\(2\)vit​a​r​g​e​t=LN⁡\(gi⊙vie​s​m\+\(1−gi\)⊙vic​t​d\)\.v\_\{i\}^\{target\}=\\mathrm\{LN\}\(g\_\{i\}\\odot v\_\{i\}^\{esm\}\+\(1\-g\_\{i\}\)\\odot v\_\{i\}^\{ctd\}\)\.\(3\)Ablation variants compare this design with ESM\-only, CTD\-only, and direct concatenation\. The gated design follows the same general motivation as gated multimodal fusion, where the network learns how much each modality contributes rather than using a fixed combination rule\[[32](https://arxiv.org/html/2609.22184#bib.bib32)\]\.

### 3\.3Cross\-Modal Interaction and Pair Features

After projection and target fusion, the drug and target vectors have the same dimensionality but remain modality\-specific\. DPTM\-DT uses a Cross\-modal Joint Interaction Module \(CJIM\) to exchange information between them\. The module follows the scaled dot\-product attention principle introduced in Transformer models and adapts it to drug\-target cross\-modal matching\[[29](https://arxiv.org/html/2609.22184#bib.bib29),[28](https://arxiv.org/html/2609.22184#bib.bib28)\]\. In one direction, the drug representation is used as the query and the target representation as key/value; in the opposite direction, the target representation queries the drug representation\. For one attention head, the drug\-to\-target update is

Ad​r​u​g←t​a​r​g​e​t=softmax⁡\(Qd​r​u​g​Kt​a​r​g​e​tTdh\)​Vt​a​r​g​e​t\.A\_\{drug\\leftarrow target\}=\\mathrm\{softmax\}\\left\(\\frac\{Q\_\{drug\}K\_\{target\}^\{T\}\}\{\\sqrt\{d\_\{h\}\}\}\\right\)V\_\{target\}\.\(4\)The reverse path is defined symmetrically\. Fig\.[2](https://arxiv.org/html/2609.22184#S3.F2)shows the two\-stream CJIM layer, where drug\-to\-target and target\-to\-drug cross\-attention are followed by residual normalization and feed\-forward updates\. The reported configuration uses two interaction layers, eight attention heads, a model dimension of 256, and a feed\-forward dimension of 1024\.

Figure 2:Structure of the Cross\-modal Joint Interaction Module\.The final pair vector concatenates four components:

zi=\[v~id​r​u​g;v~it​a​r​g​e​t;v~id​r​u​g⊙v~it​a​r​g​e​t;\|v~id​r​u​g−v~it​a​r​g​e​t\|\]\.z\_\{i\}=\[\\tilde\{v\}\_\{i\}^\{drug\};\\tilde\{v\}\_\{i\}^\{target\};\\tilde\{v\}\_\{i\}^\{drug\}\\odot\\tilde\{v\}\_\{i\}^\{target\};\|\\tilde\{v\}\_\{i\}^\{drug\}\-\\tilde\{v\}\_\{i\}^\{target\}\|\]\.\(5\)The product term describes local agreement between drug and target features, while the absolute difference term encodes feature mismatch\. Fig\.[3](https://arxiv.org/html/2609.22184#S3.F3)illustrates this construction, where the two 256\-dimensional updated modality vectors are combined into a 1024\-dimensional pair representation\. This multiplicative component is consistent with compact bilinear\-style interaction features used in multimodal representation learning\[[33](https://arxiv.org/html/2609.22184#bib.bib33)\]\. A shared multilayer perceptron mapsziz\_\{i\}into a task\-shared representation\.

Figure 3:Construction of the pair representation after cross\-modal interaction\. The updated drug and target vectors are concatenated with their element\-wise product and absolute difference to form the final drug\-target representation used by the prediction heads\.
### 3\.4Task Heads and Losses

The regression head predicts continuous affinity\. The binary head predicts whether the affinity exceeds a dataset\-specific threshold:

yib​i​n=𝕀⁡\(ai≥τ\),y\_\{i\}^\{bin\}=\\mathbb\{I\}\(a\_\{i\}\\geq\\tau\),\(6\)withτ=7\.0\\tau=7\.0for Davis pKd andτ=12\.1\\tau=12\.1for the KIBA score\. The multiclass head predicts one of six affinity levels produced by Elbow\-method\-guided K\-means discretization\. For each training split, the continuous training affinities are clustered in the one\-dimensional label space\. The number of clusters is selected by the Elbow method and set toC=6C=6for both Davis and KIBA\. The resulting cluster centers are then sorted by affinity strength, so the class index encodes an ordered low\-to\-high binding level:

yim​u​l=rank⁡\(arg⁡minc∈\{1,…,C\}​\|ai−μc\|\),y\_\{i\}^\{mul\}=\\mathrm\{rank\}\\left\(\\arg\\min\_\{c\\in\\\{1,\\ldots,C\\\}\}\|a\_\{i\}\-\\mu\_\{c\}\|\\right\),\(7\)whereμc\\mu\_\{c\}is the center of clustercc\. This construction maps a continuous regression benchmark into an affinity\-level recognition task; it does not replace the original regression objective\. As illustrated in Fig\.[4](https://arxiv.org/html/2609.22184#S3.F4), the shared pair representation is first mapped by a shared multilayer perceptron and then passed to task\-specific heads\. Training uses mean squared error for regression, binary cross\-entropy with logits for binary classification, and cross\-entropy for multiclass classification\. AdamW with warmup and cosine learning\-rate scheduling is used in training\[[30](https://arxiv.org/html/2609.22184#bib.bib30)\]\.

![Refer to caption](https://arxiv.org/html/2609.22184v1/4.png)Figure 4:Multitask prediction heads and training objectives\.

## 4Experiments and Results

### 4\.1Experimental Settings

The experiments use Davis and KIBA, two common kinase\-inhibitor benchmarks\[[26](https://arxiv.org/html/2609.22184#bib.bib26),[27](https://arxiv.org/html/2609.22184#bib.bib27)\]\. Davis contains 68 drugs, 442 targets, and 30,056 valid pairs\. Its originalKdK\_\{d\}values are converted to pKd, with a label range of 5\.00–9\.94, mean 5\.45, and standard deviation 0\.89\. KIBA contains 2,111 drugs, 229 targets, and 118,254 valid pairs\. Its score range is 0\.00–17\.20, with mean 11\.72 and standard deviation 0\.84\. Binary labels use pKd≥7\.0\\geq 7\.0for Davis and score≥12\.1\\geq 12\.1for KIBA, producing positive ratios of 8\.2% and 21\.0%, respectively\.

Table 1:Dataset, Label, and Task SummaryTable[2](https://arxiv.org/html/2609.22184#S4.T2)summarizes how the three outputs are constructed and how they can be used in a screening workflow\. The regression score supports fine\-grained ranking, the binary output supports direct high\-affinity filtering, and the six\-class output provides an interpretable affinity\-level label\. The six classes are induced by the affinity distribution of each dataset and split rather than by manually fixed intervals, which is why both Macro\-F1 and Weighted\-F1 are reported for the multiclass task\.

Table 2:Task Outputs, Label Construction, and Screening RoleTwo split protocols are reported\. The random 80/20 split supports model development and direct module comparison\. The standard split follows the DeepDTA\-style benchmark protocol and is used to reduce dependence on a single random partition\. All comparisons are organized by the same dataset and split setting whenever the corresponding baseline results are available\. The standard split is included specifically to make the comparison more referable to DeepDTA\-style benchmark protocols, while avoiding a stronger claim that every external baseline uses identical internal feature preprocessing\[[34](https://arxiv.org/html/2609.22184#bib.bib34)\]\. Regression is evaluated by MSE, RMSE,R2R^\{2\}, Pearson correlation, and concordance index \(CI\), where CI measures ranking consistency\[[37](https://arxiv.org/html/2609.22184#bib.bib37)\]\. Binary classification is evaluated by AUROC, AUPR, F1, and MCC, with AUPR and MCC being important under class imbalance\[[38](https://arxiv.org/html/2609.22184#bib.bib38),[39](https://arxiv.org/html/2609.22184#bib.bib39),[40](https://arxiv.org/html/2609.22184#bib.bib40),[35](https://arxiv.org/html/2609.22184#bib.bib35)\]\. Multiclass prediction is evaluated by Accuracy, Macro\-F1, Weighted\-F1, and Top\-2 Accuracy, which are commonly used to separate overall correctness from class\-balanced behavior\[[36](https://arxiv.org/html/2609.22184#bib.bib36)\]\.

### 4\.2Evaluation Details

For regression, the affinity labels are standardized during training and transformed back to the original scale for evaluation\. Feature and label normalization statistics are computed from the training partition only and then applied to validation and test partitions, which avoids leakage from the held\-out data\.

The model\-selection metric is task\-specific\. The regression checkpoint is selected by validation MSE, the binary checkpoint by validation AUPR, and the multiclass checkpoint by validation Macro\-F1\. This choice follows the objective of each task: value fitting for regression, positive\-pair retrieval under imbalance for binary classification, and class\-balanced performance for the six\-level task\. The classification and multiclass results are reported with mean and standard deviation, reflecting repeated runs rather than a single random draw\. Because paired per\-seed predictions are not available for all baselines, formal statistical significance is not claimed beyond the reported repeated\-run variability\[[36](https://arxiv.org/html/2609.22184#bib.bib36)\]; small numerical differences are therefore interpreted conservatively\.

The three tasks stress the representation in different ways\. Regression measures continuous affinity estimation\. Binary classification asks whether the learned representation still separates high\-affinity pairs after thresholding\. Six\-class prediction tests whether the affinity scale can be divided into ordered binding levels\. This gives a more complete view than a single regression score\.

### 4\.3Affinity Regression Results

Affinity regression is the main continuous prediction task\. Table[3](https://arxiv.org/html/2609.22184#S4.T3)reports the regression comparison on Davis and KIBA\. Under the random 80/20 split, DPTM\-DT reaches MSE/CI values of 0\.193/0\.917 on Davis and 0\.120/0\.918 on KIBA\. Under the standard split, it reports 0\.195/0\.913 on Davis and 0\.124/0\.906 on KIBA\. In both split protocols and on both datasets, DPTM\-DT has the lowest MSE among the compared methods\.

Table 3:Affinity Regression Results on Davis and KIBAThe regression pattern is consistent\. On Davis, DPTM\-DT slightly improves MSE over TEFDTA while obtaining a clearer CI advantage, which indicates better ranking of candidate pairs\. On KIBA, DPTM\-DT is close to GLCN\-DTA in MSE under the 80/20 split but has a much higher CI\. Because DTA screening often requires both accurate values and reliable ordering, this paired improvement is important\. The standard\-split results remain close to the random\-split results, suggesting that the model is not overly dependent on one favorable random split\.

The regression table also shows why reporting only one metric is incomplete\. MSE reflects value\-level error, while CI reflects whether the model preserves the relative order of affinities\. DPTM\-DT is competitive in both senses\. For example, on KIBA under the 80/20 split, the MSE difference between DPTM\-DT and GLCN\-DTA is small, but the CI difference is larger\. This indicates that the proposed representation is useful for both fitting affinity scores and prioritizing candidate pairs in virtual screening\.

### 4\.4Binary Classification Results

The binary task converts continuous affinity labels into interaction labels using the thresholds in Table[1](https://arxiv.org/html/2609.22184#S4.T1)\. This setting is more imbalanced than the regression task, especially for Davis, where positive samples are only 8\.2%\. Table[4](https://arxiv.org/html/2609.22184#S4.T4)organizes the binary results by split and dataset\.

Table 4:Binary Classification ResultsUnder the 80/20 split, DPTM\-DT obtains the highest AUROC, AUPR, F1, and MCC on both Davis and KIBA\. On Davis, AUPR increases from 0\.718 for the strongest listed baseline to 0\.727, and MCC increases from 0\.640 to 0\.654\. On KIBA, AUPR increases from 0\.782 to 0\.798\. Scores decrease under the standard split, as expected, but DPTM\-DT remains ahead of the strongest reported baselines\. The binary task is useful here because it tests high\-affinity recognition directly, rather than only the fit to continuous affinity values\.

The binary task is sensitive to class imbalance, especially on Davis, where the negative class dominates\. For this reason, AUPR is more informative than accuracy for positive\-pair retrieval\. MCC complements it by using all four entries of the confusion matrix\. DPTM\-DT improves both metrics, which indicates better positive retrieval without losing the balance between positive and negative predictions\.

### 4\.5Multiclass Classification Results

The multiclass task maps continuous affinity labels into six ordered categories\. It complements standard affinity regression by evaluating whether the model can distinguish affinity levels\. The classes are generated from the continuous label distribution by the K\-means procedure described in the Method section, with class indices ordered according to cluster centers\. This means that class frequencies are dataset\-dependent rather than manually balanced\. Table[5](https://arxiv.org/html/2609.22184#S4.T5)separates the random and standard split results and reports the main classification metrics\.

Table 5:Six\-Class Affinity\-Level Classification ResultsThe multiclass results show the same ordering trend as the regression and binary experiments\. DPTM\-DT obtains the best Macro\-F1 and Accuracy in every reported setting, reaching Macro\-F1 values of 0\.800/0\.815 under the 80/20 split and 0\.782/0\.799 under the standard split on Davis/KIBA\. KIBA is consistently higher than Davis, which matches the larger number of valid pairs and higher positive ratio in KIBA\. The reported Top\-2 Accuracy is much higher than exact Accuracy, indicating that many errors remain close to the correct affinity level\. This pattern suggests that the six\-class task mainly evaluates affinity\-level ordering and neighborhood confusion\.

The six\-class task gives another view of the continuous labels\. Macro\-F1 measures performance across affinity levels, whereas Weighted\-F1 reflects the class distribution induced by discretization\. Top\-2 Accuracy is useful because adjacent affinity bins can be hard to separate sharply\. Across both datasets, DPTM\-DT keeps the best exact and Top\-2 performance in the reported comparisons, supporting the use of one shared pair representation for both numerical prediction and coarser affinity\-level decisions\.

### 4\.6Ablation Study

Table[6](https://arxiv.org/html/2609.22184#S4.T6)summarizes representative ablations on Davis under the 80/20 split across the three tasks\. The regression ablation reports all five regression metrics for the full model and the main variants\. Binary and multiclass columns keep the metrics most directly tied to the classification objectives\.

Table 6:Representative Davis 80/20 Ablation ResultsThe ablation results separate the roles of the main modules\. ESM\-only is consistently stronger than CTD\-only, so the pretrained protein embedding is the main target\-side signal\. The full model still improves over both single\-source variants, showing that CTD statistics add useful physicochemical information\. Gated fusion is also consistently better than direct concatenation, which suggests that the target branch benefits from adaptive feature weighting\. Finally, removing cross\-attention weakens regression, binary classification, and multiclass classification, indicating that drug\-target information exchange remains useful after unimodal representation learning\.

These ablations also help interpret the relative roles of DCRM and CJIM\. DCRM improves the completeness of the target representation by combining deep sequence semantics with explicit physicochemical statistics\. CJIM then uses these representations to form a pair\-aware interaction signal\. The No\-cross\-attention variant keeps the same input features but weakens the interaction mechanism, and its drop across all three task groups indicates that the model benefits from explicit cross\-modal exchange rather than only from stronger unimodal encoders\. Although the margin size varies by metric, the direction is stable across regression, binary classification, and multiclass classification\.

### 4\.7Cross\-Task Discussion

Across the experiments, DPTM\-DT shows consistent cross\-task behavior\. The same architectural components improve continuous affinity prediction, binary interaction recognition, and affinity\-level classification\. This matters because the three tasks come from the same affinity labels but evaluate different aspects of the relation: numerical error and ranking, high\-affinity detection under imbalance, and ordinal binding\-strength levels\.

The results are also stable across split protocols\. Scores under the standard split are lower than those under the random 80/20 split, but DPTM\-DT keeps its relative advantage in every task\. KIBA generally gives stronger absolute scores than Davis, which is consistent with its larger sample size and less extreme positive\-class imbalance\.

The experimental trends match the architecture in Fig\.[1](https://arxiv.org/html/2609.22184#S3.F1)and the pair\-feature construction in Fig\.[3](https://arxiv.org/html/2609.22184#S3.F3)\. Improvements appear across tasks rather than in a single output head\. This supports the main design claim: pretrained molecular and protein representations are more effective for drug\-target prediction when they are connected through adaptive target fusion, explicit cross\-modal exchange, and task\-shared pair modeling\.

### 4\.8Practical Interpretation

The three outputs can be interpreted as a compact decision pipeline\. The regression output provides a continuous score for ranking candidate pairs\. The binary output gives a direct high\-affinity decision using dataset\-specific thresholds\. The six\-class output gives a coarser affinity\-level label that can be easier to communicate in an interface or downstream filtering workflow\. In an analysis system, a user can enter a drug SMILES string and a protein FASTA sequence and receive an interaction judgment, an affinity estimate, and an affinity\-level prediction\. This use case clarifies why the three\-task design is useful from an application perspective\.

Regression remains the primary DTA evidence because Davis and KIBA are continuous\-label benchmarks\. The binary and six\-class tasks are derived from the same labels and test whether the representation remains useful after discretization\. They enrich the evaluation, but they should be read as additional task views rather than independent datasets\.

The evaluation is centered on two widely used kinase\-inhibitor benchmarks and two split settings, which makes the results directly comparable with common DTA protocols\. This benchmark\-centered scope follows the validation practice of many drug\-target prediction studies while keeping the claims tied to measurable prediction performance and module\-level evidence\[[34](https://arxiv.org/html/2609.22184#bib.bib34),[35](https://arxiv.org/html/2609.22184#bib.bib35)\]\. Within that scope, the experiments show that the same pair representation can support value prediction, high\-affinity retrieval, and affinity\-level discrimination\.

## 5Conclusion

This paper presents DPTM\-DT, a dual\-pretrained cross\-modal multitask framework for drug\-target prediction\. The model combines GROVER drug embeddings, ESM protein embeddings, CTD target descriptors, gated target fusion, bidirectional cross\-modal attention, and task\-specific prediction heads over a shared pair representation\. Experiments cover Davis and KIBA, random 80/20 and standard splits, affinity regression, binary classification, six\-class affinity\-level classification, and Davis 80/20 ablations\. The regression head ranks candidate pairs by continuous affinity, the binary head filters high\-affinity interactions, and the six\-class head gives a coarse binding\-strength level for downstream triage\. Results favor DPTM\-DT across tasks and splits, and the ablations support the roles of dual representation, adaptive fusion, and cross\-modal interaction\. Overall, DPTM\-DT offers a practical multitask representation\-learning framework for benchmark DTA prediction and can be extended to broader target families and model\-behavior analysis\.

## References

- \[1\]M\. Bagherian, E\. Sabeti, K\. Wang, M\. A\. Sartor, Z\. Nikolovska\-Coleska, and K\. Najarian, “Machine learning approaches and databases for prediction of drug\-target interaction: a survey paper,”*Briefings in Bioinformatics*, vol\. 22, no\. 1, pp\. 247–269, 2021\.
- \[2\]Y\. Zhang, Y\. Hu, N\. Han, et al\., “A survey of drug\-target interaction and affinity prediction methods via graph neural networks,”*Computers in Biology and Medicine*, vol\. 163, p\. 107136, 2023\.
- \[3\]H\. Ozturk, A\. Ozgur, and E\. Ozkirimli, “DeepDTA: Deep drug\-target binding affinity prediction,”*Bioinformatics*, vol\. 34, no\. 17, pp\. i821–i829, 2018\.
- \[4\]H\. Ozturk, E\. Ozkirimli, and A\. Ozgur, “WideDTA: prediction of drug\-target binding affinity,”*arXiv preprint arXiv:1902\.04166*, 2019\.
- \[5\]H\. Qi, T\. Yu, W\. Yu, et al\., “Drug\-target affinity prediction with extended graph learning\-convolutional networks,”*BMC Bioinformatics*, vol\. 25, no\. 1, p\. 75, 2024\.
- \[6\]Z\. Li, P\. Ren, H\. Yang, et al\., “TEFDTA: a transformer encoder and fingerprint representation combined prediction method for bonded and non\-bonded drug\-target affinities,”*Bioinformatics*, vol\. 40, no\. 1, p\. btad778, 2024\.
- \[7\]J\. Wang, G\. Kong, and J\. Wang, “MLFF\-DTA: A multi\-level feature fusion method for predicting drug\-target binding affinity,” in*Proc\. IEEE International Conference on Bioinformatics and Biomedicine \(BIBM\)*, 2024, pp\. 751–758\.
- \[8\]J\. Wang, G\. Kong, and J\. Wang, “TriFP\-NGram: Integrating three complementary fingerprint and N\-gram features for enhanced drug\-target affinity prediction,” in*Proc\. IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*, 2025, pp\. 1–5\.
- \[9\]Z\. Wang, W\. Han, G\. Kong, and J\. Wang, “E3MMDTA: Predicting drug target affinity by combining E3FP fingerprints and multimodal fusion,” in*Proc\. IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*, 2026, pp\. 6476–6480\.
- \[10\]H\. Liu, H\. Jia, W\. Li, et al\., “KAN\-MoDTI: Drug target interaction prediction based on Kolmogorov\-Arnold network and multimodal feature fusion,”*Expert Systems with Applications*, p\. 129828, 2025\.
- \[11\]Z\. Wei, X\. Zheng, C\. Li, et al\., “DACMF\-DTI: Dual attention embedded cross\-modality fusion for drug\-target interaction prediction,”*Knowledge\-Based Systems*, vol\. 326, p\. 114063, 2025\.
- \[12\]J\. Wang, Y\. Xiao, X\. Shang, et al\., “Predicting drug\-target binding affinity with cross\-scale graph contrastive learning,”*Briefings in Bioinformatics*, vol\. 25, no\. 1, p\. bbad516, 2024\.
- \[13\]Y\. Weng, X\. Liu, H\. Li, et al\., “Drug target interaction prediction via multi\-task co\-attention,”*International Journal of Data Mining and Bioinformatics*, vol\. 24, no\. 2, pp\. 160–176, 2020\.
- \[14\]W\. Zhao, Y\. Yu, G\. Liu, et al\., “MSI\-DTI: Predicting drug\-target interaction based on multi\-source information and multi\-head self\-attention,”*Briefings in Bioinformatics*, vol\. 25, no\. 3, p\. bbae238, 2024\.
- \[15\]L\. Wang, Y\. Zhou, and Q\. Chen, “AMMVF\-DTI: A novel model predicting drug\-target interactions based on attention mechanism and multi\-view fusion,”*International Journal of Molecular Sciences*, vol\. 24, no\. 18, p\. 14142, 2023\.
- \[16\]Y\. Feng, X\. Chen, J\. Guo, et al\., “KG\-MACNF: A nonlinear cross\-modal fusion model for drug\-target interaction prediction based on knowledge graph and multi\-level attention mechanism,”*PLOS ONE*, vol\. 20, no\. 9, p\. e0331037, 2025\.
- \[17\]Y\. Zhang, Y\. Wang, C\. Wu, et al\., “Drug\-target interaction prediction by integrating heterogeneous information with mutual attention network,”*BMC Bioinformatics*, vol\. 25, no\. 1, p\. 361, 2024\.
- \[18\]J\. Jiang, L\. Chen, L\. Ke, et al\., “A review of transformer models in drug discovery and beyond,”*Journal of Pharmaceutical Analysis*, vol\. 15, no\. 6, p\. 101081, 2025\.
- \[19\]Y\. Rong, Y\. Bian, T\. Xu, et al\., “Self\-supervised graph transformer on large\-scale molecular data,” in*Advances in Neural Information Processing Systems*, vol\. 33, pp\. 12559–12571, 2020\.
- \[20\]Z\. Lin, H\. Akin, R\. Rao, et al\., “Evolutionary\-scale prediction of atomic\-level protein structure with a language model,”*Science*, vol\. 379, no\. 6637, pp\. 1123–1130, 2023\.
- \[21\]J\. Delile, S\. Mukherjee, J\. Mueller, et al\., “Foundation models in drug discovery: Phenomenal growth today, transformative potential tomorrow?”*Drug Discovery Today*, vol\. 30, p\. 104518, 2025\.
- \[22\]R\. Sheshanarayana and F\. You, “Molecular representation learning: cross\-domain foundations and future frontiers,”*Digital Discovery*, vol\. 4, no\. 9, pp\. 2298–2335, 2025\.
- \[23\]G\. Kong, J\. Wang, and J\. Wang, “GATOmics: A novel multi\-omics graph attention network model for cancer driver gene detection,” in*Proc\. IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*, 2025, pp\. 1–5\.
- \[24\]G\. Kong, J\. Wang, Y\. Fan, and J\. Wang, “SLP\-T5: A new method for predicting protein sub\-chloroplast localization based on text\-to\-text transfer transformer,” in*Proc\. IEEE International Conference on Bioinformatics and Biomedicine \(BIBM\)*, 2024, pp\. 185–190\.
- \[25\]G\. Kong, Y\. Fan, J\. Wang, and Z\. Yang, “Messenger RNA subcellular localization prediction via large language models and attention mechanisms,” in*Proc\. IEEE International Conference on Systems, Man, and Cybernetics \(SMC\)*, 2024, pp\. 4751–4756\.
- \[26\]M\. I\. Davis, J\. P\. Hunt, S\. Herrgard, et al\., “Comprehensive analysis of kinase inhibitor selectivity,”*Nature Biotechnology*, vol\. 29, no\. 11, pp\. 1046–1051, 2011\.
- \[27\]J\. Tang, A\. Szwajda, S\. Shakyawar, et al\., “Making sense of large\-scale kinase inhibitor bioactivity data sets: a comparative and integrative analysis,”*Journal of Chemical Information and Modeling*, vol\. 54, no\. 3, pp\. 735–743, 2014\.
- \[28\]Y\.\-H\. H\. Tsai, S\. Bai, P\. P\. Liang, J\. Z\. Kolter, L\.\-P\. Morency, and R\. Salakhutdinov, “Multimodal transformer for unaligned multimodal language sequences,” in*Proc\. 57th Annual Meeting of the Association for Computational Linguistics*, 2019, pp\. 6558–6569\.
- \[29\]A\. Vaswani, N\. Shazeer, N\. Parmar, et al\., “Attention Is All You Need,” in*Advances in Neural Information Processing Systems*, vol\. 30, pp\. 5998–6008, 2017\.
- \[30\]I\. Loshchilov and F\. Hutter, “Decoupled Weight Decay Regularization,” in*Proc\. International Conference on Learning Representations \(ICLR\)*, 2019\.
- \[31\]I\. Dubchak, I\. Muchnik, S\. R\. Holbrook, and S\.\-H\. Kim, “Prediction of protein folding class using global description of amino acid sequence,”*Proceedings of the National Academy of Sciences*, vol\. 92, no\. 19, pp\. 8700–8704, 1995\.
- \[32\]J\. Arevalo, T\. Solorio, M\. Montes\-y\-Gomez, and F\. A\. Gonzalez, “Gated multimodal networks,”*Neural Computing and Applications*, vol\. 32, no\. 14, pp\. 10209–10228, 2020\.
- \[33\]J\.\-H\. Kim, K\.\-W\. On, W\. Lim, J\. Kim, J\.\-W\. Ha, and B\.\-T\. Zhang, “Hadamard Product for Low\-rank Bilinear Pooling,” in*Proc\. International Conference on Learning Representations \(ICLR\)*, 2017\.
- \[34\]Z\. Tanoli, A\. Schulman, and T\. Aittokallio, “Validation guidelines for drug\-target prediction methods,”*Expert Opinion on Drug Discovery*, vol\. 20, no\. 1, pp\. 31–45, 2025\.
- \[35\]M\. Ghanem, A\. K\. Ghaith, V\. G\. El\-Hajj, et al\., “Limitations in evaluating machine learning models for imbalanced binary outcome classification in spine surgery: a systematic review,”*Brain Sciences*, vol\. 13, no\. 12, p\. 1723, 2023\.
- \[36\]O\. Rainio, J\. Teuho, and R\. Klen, “Evaluation metrics and statistical tests for machine learning,”*Scientific Reports*, vol\. 14, no\. 1, p\. 6086, 2024\.
- \[37\]F\. E\. Harrell, R\. M\. Califf, D\. B\. Pryor, K\. L\. Lee, and R\. A\. Rosati, “Evaluating the yield of medical tests,”*JAMA*, vol\. 247, no\. 18, pp\. 2543–2546, 1982\.
- \[38\]B\. W\. Matthews, “Comparison of the predicted and observed secondary structure of T4 phage lysozyme,”*Biochimica et Biophysica Acta \(BBA\)\-Protein Structure*, vol\. 405, no\. 2, pp\. 442–451, 1975\.
- \[39\]J\. Davis and M\. Goadrich, “The relationship between Precision\-Recall and ROC curves,” in*Proc\. 23rd International Conference on Machine Learning*, 2006, pp\. 233–240\.
- \[40\]T\. Saito and M\. Rehmsmeier, “The precision\-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets,”*PLOS ONE*, vol\. 10, no\. 3, p\. e0118432, 2015\.