arXiv:2605.30247v1 [cs.LG] 28 May 2026
OOD-GraphLLM: Graph Large Language Model for Out-of-Distribution Generalized Drug Synergy Prediction Xin Wang
Linxin Xiao
DCST, BNRist, Tsinghua University Beijing, China [email protected]
DCST, Tsinghua University Beijing, China [email protected]
Yang Yao
Wenwu Zhu∗
DCST, Tsinghua University Beijing, China [email protected]
DCST, BNRist, Tsinghua University Beijing, China [email protected]
Abstract Drug synergy prediction (DSP) aims to identify efficacious drug combinations under various cellular contexts with different targets. However, the continual emergence of novel compounds results in variations in molecular scaffolds and sizes, causing drug synergy data to exhibit out-of-distribution (O.O.D.) shifts with respect to topological structure. Existing works rely on in-distribution (I.D.) assumption, failing to handle the O.O.D. shifts. To solve this problem, we study out-of-distribution generalized drug synergy prediction through a graph large language model for the first time. Nevertheless, O.O.D. generalized DSP is highly non-trivial, posing several challenges: i) how to discover structurally relevant and irrelevant molecular representations with respect to cell targets; ii) how to find the optimal graph neural architectures that accurately calculate molecular representations; and iii) how to jointly leverage molecular structural and semantic information in LLMs. To address these challenges, we propose OOD-GraphLLM, a novel graphLLM framework which is able to accurately predict drug synergy under O.O.D. settings via jointly optimizing molecular graph representation and biomedical semantic language representations in a unified manner. Concretely, we first propose a target-adaptive disentangled molecular graph encoding model to distinguish targetrelevant and target-irrelevant molecular representations for both seen and unseen drugs, then introduce a pairwise attentive graph architecture search algorithm that dynamically finds the best neural architectures to calculate molecular representations for different and new drug pairs, followed by our design of multi-level contextualized cellular feature alignment mechanism to incorporate cell line context information at both structural and semantic levels. Furthermore, we finetune DrugSyn-LLM, a biomedical LLM, and employ ∗ Corresponding author. DCST is the abbreviation for Department of Computer Science
and Technology. BNRist is the abbreviation for Beijing National Research Center for Information Science and Technology. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. Conference’17, Washington, DC, USA © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-x-xxxx-xxxx-x/YYYY/MM https://doi.org/10.1145/nnnnnnn.nnnnnnn
a retrieval-augmented biomedical instruction tuning strategy to align molecular topological information and molecular semantic information with language-based reasoning for O.O.D. generalized DSP. Extensive experiments under several O.O.D. settings demonstrate that the proposed OOD-GraphLLM consistently outperforms state-of-the-art approaches on various DSP tasks. Both the source code 1 and released model 2 are publicly available, where users are allowed to download model resources and interactively use the system through a web interface.
CCS Concepts • Applied computing → Bioinformatics; • Computing methodologies → Neural networks.
Keywords Graph Neural Network (GNN), Large Language Model (LLM), Drug Synergy Prediction (DSP), Out-of-distribution (O.O.D.) ACM Reference Format: Xin Wang, Linxin Xiao, Yang Yao, and Wenwu Zhu. 2026. OOD-GraphLLM: Graph Large Language Model for Out-of-Distribution Generalized Drug Synergy Prediction. In . ACM, New York, NY, USA, 12 pages. https://doi. org/10.1145/nnnnnnn.nnnnnnn
1
Introduction
Drug combination therapy [9] has emerged as a promising strategy for treating complex diseases such as cancer and drug-resistant infections. Compared to single-drug treatments, effective drug combinations can enhance therapeutic efficacy, reduce toxicity, and mitigate the development of resistance. Therefore, Drug synergy prediction (DSP) [15, 20, 24, 31] has become a critical and fundamental problem in computational drug discovery, aiming to identify efficacious drug combinations under various cellular contexts with different cell targets. Recent advances in machine learning, particularly graph neural networks (GNNs) [22, 32, 37], have substantially improved DSP by conducting representation learning over the molecular topological structures of drugs to predict drug-drug synergies. By representing drugs as molecular graphs, existing literature is able to capture topological structural information that 1 https://github.com/EkkoXiao/Bio-GraphLLM 2 https://mn.cs.tsinghua.edu.cn/bio-graphllm/
Conference’17, July 2017, Washington, DC, USA
Xin Wang et al.
• We are the first to study the problem of out-of-distribution generalized drug synergy prediction (O.O.D. generalized DSP) by resorting to graph large language models, to the best of our knowledge. • We propose OOD-GraphLLM, a novel graphLLM framework for accurate O.O.D. generalized DSP, which jointly optimizes our proposed four components, i.e., (1) target-adaptive disentangled molecular graph encoding, (2) pairwise attentive graph architecture search, (3) multi-level contextualized cellular feature alignment and (4) finetuned biomedical LLM DrugSyn-LLM with retrieval-augmented instruction tuning, within a unified framework. • We conduct extensive experiments under multiple O.O.D. evaluation settings to demonstrate that the proposed OOD-GraphLLM is able to consistently outperform state-of-the-art baselines, highlighting its superior generalization ability and prediction accuracy under molecular topological distribution shifts.
plays an important role in determining the corresponding molecular chemical properties. However, due to the continuous emergence of novel compounds in drug discovery, drug synergy data often exhibit out-of-distribution (O.O.D.) shifts with respect to topological structure at the drug level, primarily caused by variations in molecular scaffolds and compound sizes. This out-of-distribution generalized drug synergy prediction (O.O.D. generalized DSP) problem requires models to generalize well to new drugs and previously unseen molecular scaffolds. Fig. 1 (a) illustrates one example of O.O.D. generalized DSP. Existing works on DSP heavily rely on the in-distribution (I.D.) assumption, where the drug structures tend to remain the same for training and testing data, failing to handle the O.O.D. shifts. In this paper, we study out-of-distribution generalized drug synergy prediction by resorting to a graph large language model (shown in Fig. 1 (c)), to the best of our knowledge, for the first time. Nevertheless, O.O.D. generalized DSP is highly non-trivial, with several key challenges. For drugs with O.O.D. topological structures,
Out-Of-Distribution (O.O.D.)
In-Distribution (I.D.)
(1) it is challenging to obtain structurally relevant and irrelevant molecular representations with respect to cell targets; (2) it is challenging to find the optimal graph neural architecture that can calculate accurate molecular representations; (3) it is challenging to jointly leverage structural and semantic information from molecules within LLMs. To address these challenges, we propose OOD-GraphLLM, a novel graphLLM framework capable of accurately predicting drug synergy under O.O.D. setting via joint optimization of molecular graph representation and biomedical semantic language representations in a unified manner. Given new drugs, the proposed OODGraphLLM treats cell lines as contexts, obtains the best graph neural network (GNN) architecture for calculating the new molecular graph representations, aligns these representations with both topological and semantic cellular features, tokenizes all the features as input to a biomedically finetuned LLM for O.O.D. generalized DSP, and optimizes the whole procedure within one single framework. In concrete, we first propose a target-adaptive disentangled molecular graph encoding model that learns target-relevant and target-irrelevant molecular representations, conditioning the target-relevant representations on various cell targets with disentanglement constraint to preserve target-aware information for new molecular structures. Building upon the target-adaptive disentangled molecular graph representations, we introduce a pairwise attentive graph architecture search algorithm that dynamically finds the optimal graph neural architectures for new drug pairs, allowing for accurate molecular representation learning under distribution shifts. We then design a multi-level contextualized cellular feature alignment mechanism to incorporate cell line information into molecular graph representations as contexts at both the structural and semantic levels. Last but not least, we finetune DrugSyn-LLM, a biomedical LLM, with retrieval-augmented biomedical instruction tuning strategy, aligning molecular topological and semantic information with language-based reasoning to accomplish O.O.D. generalized DSP. We conduct extensive experiments to demonstrate the superiority of OOD-GraphLLM over state-of-the-art baselines under various O.O.D. settings. The contributions of this paper are summarized as follows:
TCS 401
(a) O.O.D. generalized DSP c1cc2c(s1)CNCC problem
Fingolimod
Topological Shift c1ccccc1
Colchicine
Mitomycin C
O=C1C=CC(=O)C2= C1CC1C3NC3CN21
O=c1cccc2c(c1)CCC c1ccccc1-2
Prediction for New Drugs
Training on Existing Drugs
Ring Count, Bertz Complexity Index ↑ DrugSyn-LLM Knowledge Retrieval
Contextualized Embedding
Fixed Neural Architecture
Adaptive Neural Architecture
Handcrafted Features Cell Line
Target-Adaptive Disentangled Features
New Drugs
(b) Current methods
Cell Line
V.S
Target
New Drugs
(c) OOD-GraphLLM
Figure 1: Comparisons between current methods (b) and OODGraphLLM (c) under O.O.D. generalized DSP (a) settings.
2
Related Works
Drug Synergy Prediction (DSP). Deep learning-based models have been widely adopted for drug synergy prediction due to their ability to model complex drug–drug and drug–cell interactions. Early methods, such as DeepSynergy [31], MatchMaker [20], and TreeCombo [15], mainly used molecular descriptors and cellline gene expression profiles with deep neural networks or gradient boosting models. With the development of graph neural networks (GNNs), methods such as DeepDDS [37], DDoS [32], and GAECDS [22] represented drugs as molecular graphs to capture atom-level structural information. Meanwhile, DFFNDDS [41] exploited pretrained language models such as BERT [4] to extract semantic representations from SMILES strings, while DTSyn [11] and AttenSyn [38] further introduced attention-based Transformer architectures for drug–drug and drug–cell interaction modeling. More recently, BAITSAO [29] has explored large language models as predictors for drug synergy tasks. Despite these advances, most existing methods are developed under the in-distribution assumption and are rarely designed for O.O.D. generalization, leaving the
OOD-GraphLLM: Graph Large Language Model for Out-of-Distribution Generalized Drug Synergy Prediction
integration of GNNs and LLMs for distribution-shift-aware drug synergy prediction underexplored. Graph Large Language Models (GraphLLMs). Graph large language models (GraphLLMs) extend the reasoning and generation abilities of LLMs to graph-structured data, enabling tasks such as graph understanding and question answering [12, 17, 39]. Existing studies mainly follow two directions. Prompt-based methods, such as InstructGLM [43] and NLGraph [36], translate graph structures into textual prompts that can be interpreted by LLMs. Representation-alignment methods, such as GraphGPT [34] and GraphLLM [2], encode graphs with GNNs and feed the resulting graph tokens into language models. Further works, including GLEM [45] and PATTON [18], explore iterative co-training and alignment between GNNs and LLMs to improve representation learning. Motivated by the graph structure of molecules and the semantic information in SMILES, recent studies such as MolTC [7] and DyNAS-DDI [40] have applied GraphLLM-style architectures to drug-related tasks. However, existing GraphLLM frameworks are mostly designed for constrained prediction or reasoning scenarios, and have not fully addressed complex drug synergy prediction settings that require modeling higher-order drug–drug–cell context and O.O.D. shifts.
3
OOD-GraphLLM
In this section, we describe the proposed OOD-GraphLLMin detail. Sec 3.1 formally formulates the drug synergy prediction problem and defines the learning objectives. Sec 3.2 and Sec 3.3 present the target-adaptive disentangled molecular graph encoding model and the pairwise attentive graph architecture search algorithm, respectively. Sec 3.4 describes the multi-level contextualized cellular feature alignment mechanism, and finally, Sec 3.5 details the design of finetuning DrugSyn-LLM with the retrieval-augmented biomedical instruction tuning strategy and outlines the overall multi-stage training procedure. Fig. 2 shows the overall framework of OOD-GraphLLM.
3.1
Problem Formulation
Drug synergy prediction (DSP) aims to characterize the combined effect of multiple drugs under a specific cell line context. Take the most common setting, i.e., DSP for two drugs, as an example, we formally define DSP as follows. Let D = {𝑑 1, 𝑑 2, . . . , 𝑑𝑛 } denote the universe of drug molecules and let C = {𝑐 1, 𝑐 2, . . . , 𝑐𝑚 } denote the set of cell lines representing distinct biological environments. Each data instance is characterized by a triplet (𝑑𝑖 , 𝑑 𝑗 , 𝑐𝑘 ), where 𝑑𝑖 , 𝑑 𝑗 ∈ D are two drugs administered in combination and 𝑐𝑘 ∈ C specifies the cellular condition. The learning objective is to infer a predictive function 𝑓 as follows: 𝑓 : (𝑑𝑖 , 𝑑 𝑗 , 𝑐𝑘 ) ↦→ (𝑦𝑖𝑘𝑗 , 𝑠𝑖𝑘𝑗 ),
(1)
where 𝑦𝑖𝑘𝑗 ∈ Y denotes a discrete interaction label, and 𝑠𝑖𝑘𝑗 ∈ R represents a continuous synergy score quantifying the strength of the combinatorial effect. Out-of-distribution generalized drug synergy prediction (O.O.D. generalized DSP) studies generalization beyond observed drug distributions. Specifically, the drug space D is partitioned into an in-distribution subset DI.D. and an out-of-distribution subset
Conference’17, July 2017, Washington, DC, USA
DO.O.D. , according to criteria such as molecular scaffolds and sizes etc. These subsets satisfy DI.D. ∩DO.O.D. = ∅ and DI.D. ∪DO.O.D. = D. Under this protocol, the training set solely consists of drugs drawn from DI.D. , while validation and test sets include drug pairs where at least one drug belongs to DO.O.D. . Dtrain = { (𝑑𝑖 , 𝑑 𝑗 , 𝑐𝑘 ) | 𝑑𝑖 , 𝑑 𝑗 ∈ DI.D. , 𝑐𝑘 ∈ C }, Dvalid ∪ Dtest = { (𝑑𝑖 , 𝑑 𝑗 , 𝑐𝑘 ) | 𝑑𝑖 ∈ DO.O.D. ∨ 𝑑 𝑗 ∈ DO.O.D. , 𝑐𝑘 ∈ C }. (2)
3.2
Target-Adaptive Disentangled Molecular Graph Encoding
To capture both intrinsic molecular topological structures and target-dependent characteristics, we propose the target-adaptive disentangled molecular graph encoding model. We first learn targetrelevant and target-irrelevant molecular representations through disentangled molecular graph encoding, then condition the targetrelevant representations on various cell targets through cross-attention with associated target proteins by target-adaptive representation learning. To further encourage target-adaptive disentanglement, we explicitly impose a decorrelation constraint on the conditioned target-relevant representations so that they can well preserve different target-specific information. Disentangled Molecular Graph Encoding. Each drug molecule 𝑑 is represented as a molecular graph G𝑑 = (V𝑑 , E𝑑 ) where V𝑑 and E𝑑 denote the node set and edge set respectively. For each node 𝑣 ∈ V𝑑 , we extract a set of node-level representations from 𝑀 heterogeneous GNNs, and aggregate them into a graph-level vector: zG𝑑 = Φpool
∑︁
Ψ GNN1 (𝑣), . . . , GNN𝑀 (𝑣) ,
(3)
𝑣 ∈V𝑑
where Ψ(·) denotes a feature fusion operator over multiple GNN views, and Φpool (·) represents the pooling function. The resulting embedding z G𝑑 ∈ R𝐷 captures structural and chemical characteristics of the molecular graph. To separate intrinsic molecular characteristics from target-dependent characteristics, we introduce a disentanglement head that decomposes z G𝑑 into target-irrelevant representations and target-relevant representations: z𝑑irr = Wirr zG𝑑 ,
z𝑑rel = Wrel zG𝑑 ,
(4)
where z𝑑irr ∈ R𝐷 irr captures intrinsic properties of the molecular graph that are independent of specific target protein instantiations, while z𝑑rel ∈ R𝐷 rel carries the target-relevant information, with 𝐷 irr + 𝐷 rel = 𝐷. Target-Adaptive Representation Learning. Target proteins play a critical role in drug synergy prediction, as they mediate the molecular mechanisms through which drugs exert their effects. To incorporate target-specific biological context, we further condition z𝑑rel 𝐾 denote the embeddings on drug-associated targets. Let {t (𝑘 ) }𝑘=1 of the 𝐾 corresponding targets related to drug 𝑑, obtained from a pretrained protein encoder ESM-2 [27]. We apply a cross-attention 𝐾 to produce target-adaptive mechanism between z𝑑rel and {t (𝑘 ) }𝑘=1 representations: (𝑘 )
z̃𝑑
= CrossAttn z𝑑rel , t (𝑘 ) ,
𝑘 = 1, . . . , 𝐾 .
(5)
The final molecular representations are formed by concatenating the target-irrelevant representations with all target-adaptive
Conference’17, July 2017, Washington, DC, USA
Xin Wang et al.
(1) Target-Adaptive Disentangled Molecular Graph Encoding Targets
Decorrelation Layer
Cross Attn
ESM-2
Graph Neural Network
Input
ESM-2
Targets
Disentangle Head
Input
𝐸𝐸1
𝐸𝐸1
Attention Mechanism 𝐸𝐸�
𝐸𝐸2
𝐻𝐻1
𝐸𝐸2
Attn
Decorrelation Layer
Input
Cellular Context Features
Gene Expression Profiles Molecular Graphs
Concatenate
Drug-based Retrieval Biomedical Repositories Task Training
Attention-based Pairwise Router Operation Vectors
Drug Info Drug Info
Instruction-Tuning
Q: What are the Q: Do the two drugs [Property Description] exhibit synergy? of #Drug1 and What is their [Score]? #Drug2? A: Yes./No. Their A: [#Drug1 Info] score is [Score]. [#Drug2 Info]
Cosine Similarities
Molecular Graphs Cell Line
𝐻𝐻1
𝐸𝐸2 ||𝐸𝐸�
𝑳𝑳𝒔𝒔𝒔𝒔𝒔𝒔
𝑳𝑳𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅,𝟐𝟐
Cross
𝐸𝐸1 ||𝐸𝐸�
+(𝟏𝟏 − 𝜹𝜹) 𝑳𝑳𝒕𝒕𝒕𝒕𝒕𝒕𝒕𝒕 𝜹𝜹 ∈ {𝟎𝟎, 𝟏𝟏}
(4) Retrieval-augmented Biomedical Instruction Tuning
(2) Pairwise Attentive Graph Architecture Search
𝑳𝑳𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅,𝟏𝟏
Input
𝒊𝒊𝒊𝒊𝒊𝒊𝒊𝒊
𝑀𝑀𝑀𝑀 1𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐
op1,1
op1,2 op2,2 op1,3Attention-based op2,3 Pairwise Router Customized GNN Architecture
Contextualized Molecular Graphs
𝑀𝑀𝑀𝑀 2𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐
op1,1
op2,1
op1,2
op2,2
op1,3
op2,3
Semantic Level
Structural Level
Flattening
op2,1
A2058 is a cell exhibiting epithelial
morphology isolated from the skin...
Drug 2 SMILES
(3) Multi-Level Contextualized Cellular Feature Alignment
Mapping
Q/A Prompts
Q-Former
Operation Weights
Query Tokens
Molecular Representations
Q-Former Drug Projector
Operation Weights
Cell Representations
Query Tokens
Q-Former
Cell Line Text Description
Cell Projector
Drug 1 SMILES
Tokenizer
Drug Syn - L L M
Input
𝑳𝑳𝑳𝑳𝑳𝑳𝑳𝑳 = 𝜶𝜶( 𝑳𝑳𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅,𝟏𝟏 + 𝑳𝑳𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅,𝟐𝟐 ) + 𝜷𝜷 𝑳𝑳𝒔𝒔𝒔𝒔𝒔𝒔
OOD-GraphLLM: Total Loss + 𝜹𝜹 𝑳𝑳
Drug Synergy Prediction
𝑳𝑳𝒊𝒊𝒊𝒊𝒊𝒊𝒊𝒊
𝑳𝑳𝒕𝒕𝒕𝒕𝒕𝒕𝒕𝒕
Figure 2: The overall framework of OOD-GraphLLM . OOD-GraphLLM is able to conduct accurate O.O.D. generalized DSP by integrating cellular contextualized molecular graph representation learning and LLMs together through four jointly optimized components, i.e., (1) Target-Adaptive Disentangled Molecular Graph Encoding; (2) Pairwise Attentive Graph Architecture Search; (3) Multi-Level Contextualized Cellular Feature Alignment; and (4) Finetuned DrugSyn-LLM with Retrieval-Augmented Biomedical Instruction Tuning. representations, h
(1)
(𝐾 )
e𝑑 = z𝑑irr ∥ z̃𝑑 ∥ · · · ∥ z̃𝑑
i .
(6)
To further encourage disentanglement across different target conditioned representations, we impose a decorrelation constraint directly on the set of target-adaptive representations. Specifically, for 𝐾 , we explicitly penalize 𝐾 target-adaptive representations {z̃𝑑(𝑘 ) }𝑘=1 statistical dependencies between every pair of these representations. For (𝑘, 𝑘 ′ ) with 𝑘 ≠ 𝑘 ′ , we first normalize the representations within a mini-batch and compute their cross-correlation matrix: C𝑘,𝑘 ′ =
1 (𝑘 ) (𝑘 ′ ) ⊤ z̃ z̃ , 𝐷𝑘
(7)
where 𝐷𝑘 denotes the dimensionality of each target-adaptive representation. The decorrelation loss is then defined as the average squared Frobenius norm of the cross-correlation matrices over all pairs: Ldecorr =
∑︁ 1 2 C𝑘,𝑘 ′ 𝐹 . 𝐾 (𝐾 − 1) ′
and target-relevant representations under different cell target contexts, and plays a critical role in improving generalization under drug-level O.O.D. settings.
3.3
Pairwise Attentive Graph Architecture Search
Given new pairs of drugs, we propose the pairwise attentive graph architecture search algorithm to discover the optimal GNN architectures for calculating drug molecular graph representations accurately. There are three core parts for completing this task: i) molecular pairwise attention that injects bidirectional drug pair context into molecular graph representations, ii) latent operator space parameterization that projects candidate message-passing operators into a continuous and differentiable latent space, and iii) adaptive routing for architecture search that dynamically selects and assembles the most appropriate operators based on molecular graph representations in the projected latent space.
(8)
𝑘≠𝑘
By minimizing Ldecorr , the model is encouraged to learn representations that encode distinct aspects of target-relevant drug information, thereby reducing redundancy in representation spaces. This design enforces functional disentanglement across target-irrelevant
Molecular Pairwise Attention. Let e𝑑1 and e𝑑2 denote molecular graph representations of the two drugs (i.e., 𝑑 1 and 𝑑 2 ) obtained in Sec 3.2. To incorporate the influence of one molecule on the other, we introduce a bidirectional drug pair context injection module based on multi-head attention. Specifically, the representation of
OOD-GraphLLM: Graph Large Language Model for Out-of-Distribution Generalized Drug Synergy Prediction
drug 𝑑 1 is expressed as follows: h𝑑1 = e𝑑1 + FFN Amh (Q, K, V) ,
(9)
where the query is constructed by Q = W𝑄 e𝑑1 , while the keyvalue pairs are derived from 𝑑 2 as K, V = W𝐾,𝑉 e𝑑2 . An analogous operation is applied symmetrically to obtain h𝑑2 . Building upon the target-adaptive disentangled molecular graph representations from Sec 3.2, this attention mechanism allows the pairwise attentive representations to encode pharmacophoric signals that are critical for extrapolating to drug combinations under distribution shifts. Latent Operator Space Parameterization. To support pairwise attentive architectural customization while preserving differentiable trainability, we introduce a latent parameterization of candidate message-passing operators. At each GNN layer 𝑙, we maintain a (𝑙 ) collection of aggregation primitives O (𝑙 ) = {op1(𝑙 ) , op2(𝑙 ) , . . . , op𝑚 }, where each primitive encodes a distinct strategy for information propagation in molecular graph. We project them into a shared latent operator space by associating each operator op𝑖(𝑙 ) with a learn(𝑙 ) able vector o𝑖(𝑙 ) ∈ R𝑑op . The resulting set E (𝑙 ) = {o1(𝑙 ) , . . . , o𝑚 } defines a continuous operator space that enables smooth interpolation between aggregation patterns. To ensure that different operators remain functionally distinguishable within this latent space, we explicitly discourage excessive similarity among their representations. Specifically, we introduce a layerwise separation constraint that penalizes high cosine similarity between distinct operator descriptors:
from over-specializing to spurious correlations observed in the training distribution. As a result, the learned representations x𝑑1 and x𝑑2 capture not only intrinsic molecular structures, but also pair-aware structural adaptations that are capable of generalization to O.O.D. settings for new drugs.
3.4
(𝑙 ) ∑︁ © o (𝑙 ) o𝑗 1 ª 𝑖 (𝑙 ) · ® . 𝑚 (𝑚 − 1) 𝑖≠𝑗 ∥o ∥ 2 ∥o (𝑙 ) ∥ 2 𝑗 « 𝑖 ¬
Structural Level Feature Alignment. We utilize cell line gene expression profiles to capture context-dependent atomic interactions when learning molecular graph representation. Given a cell line 𝑐, let x𝑐 ∈ R𝐷𝑐 denote its gene expression feature vector, which is transformed into context representations e𝑐 via a projection function: e𝑐 = 𝜙 ctx (x𝑐 ). For drug 𝑑 characterized via a molecular graph G𝑑 = (V𝑑 , E𝑑 ), each atom in drug 𝑑 can be described by node 𝑣 ∈ V𝑑 , which is associated with an initial atomic feature vector z𝑣 . We augment the molecular graph representation by concatenating the context representations with atomic feature vectors as follows: z̃𝑣 = [z𝑣 ∥ e𝑐 ] ,
(10)
This regularization term softly enforces angular separation among operator representations, preventing the latent operator space from collapsing into a low-rank configuration. The overall Lsep is ob(𝑙 ) tained by averaging Lsep across all layers. Adaptive Routing for Architecture Search. Given the drug representation h𝑑 , we modulate the contribution of each candidate operator at layer 𝑙 through an adaptive routing mechanism. The routing weight assigned to operator op𝑖(𝑙 ) is computed as follows: (𝑙 ) exp ⟨h𝑑 , o𝑖 ⟩ (𝑙 ) , 𝛼𝑖 = Í (𝑙 ) 𝑚 𝑗 =1 exp ⟨h𝑑 , o 𝑗 ⟩
(𝑙 +1)
=
𝑚 ∑︁
(𝑙 )
𝛼𝑖
(𝑙 )
op𝑖
(𝑙 )
x𝑑
,
(0)
x𝑑 = z̃𝑣 ,
∀𝑣 ∈ V𝑑 .
(13)
This results in a contextualized molecular graph G𝑑𝑐𝑒𝑙𝑙 , in which the atom (i.e., node) representations contain the cellular contexts. Such structural level contextualization allows downstream graph encoders to take the influence of cell specific biological contexts into consideration. Semantic Level Feature Alignment. In addition to structural level concatenation, we further align cellular information at the semantic level by projecting cell line descriptions into the input space of the LLM. Specifically, given textual descriptions of cell line 𝑐 together with its gene expression vector x𝑐 , we employ a tokenizer and a BERT-based projector [23] to obtain a set of cell specific tokens and representations:
(11) T𝑐 = Tokenizer(𝑐 ),
where ⟨·, ·⟩ denotes the dot-product similarity. These routing weights can be used to select the best operators for assembling the optimal graph neural architecture. As such, molecular graph representations of the corresponding drugs can be computed through the optimal graph neural network as follows: x𝑑
Multi-Level Contextualized Cellular Feature Alignment
Drug synergy is inherently cell line dependent, as cellular environments determine drug sensitivity, pathway activation, and synergistic effect. To explicitly incorporate cellular context into drug representations and LLM finetuning, we introduce the multi-level contextualized cellular feature alignment mechanism. At the structural level, molecular graph representations are augmented with cell line information in the form of context feature concatenation. At the semantic level, cellular textual descriptions and gene expression profiles are utilized to align with the LLM input space.
2
(𝑙 ) Lsep =
Conference’17, July 2017, Washington, DC, USA
E𝑐 = 𝜙 proj𝑐 (x𝑐 ),
(14)
where T𝑐 denotes the discrete cell tokens and E𝑐 represents continuous cell representations aligned with the input space of the LLM. By jointly leveraging structural and semantic level representation contextualization, we are able to align cellular contexts to molecular graphs and language models simultaneously, which is a critical support for O.O.D. generalized DSP for new drugs.
(12)
𝑖=1
where x𝑑(𝑙+1) denotes the computed molecule representation at layer 𝑙. This algorithm allows operator parameters and routing weights to be jointly optimized during finetuning, effectively inducing pairaware computational graphs. By conditioning operator routing on pairwise attentive molecular graph representations, this adaptive mechanism prevents the model
3.5
DrugSyn-LLM with Retrieval-Augmented Biomedical Instruction Tuning
We finetune DrugSyn-LLM, our biomedical LLM, with the proposed Retrieval-Augmented Biomedical Instruction Tuning strategy to inject various domain knowledge into LLM while enabling taskspecific reasoning. Given an input prompt P, the tokenized textual
Conference’17, July 2017, Washington, DC, USA
Xin Wang et al.
descriptions of the cell line T𝑐 , its projected representations E𝑐 , the tokenized SMILES sequences of two drugs TSMILES1 and TSMILES2 , DrugSyn-LLM aims to produce an appropriate response 𝑅 with accurate prediction. To bridge molecular graph representations and natural language reasoning, the drug representations x𝑑1 and x𝑑2 obtained in Sec 3.4 are first projected into the language-aligned space via 𝜙 proj𝑑 , which adopts the same BERT-based architecture as 𝜙 proj𝑐 but is parameterized independently, yielding E𝑑1 = 𝜙 proj𝑑 (x𝑑1 ) and E𝑑2 = 𝜙 proj𝑑 (x𝑑2 ). The overall training procedure consists of two stages: biomedical instruction tuning and task-specific training.
Table 1: O.O.D. Dataset Splitting Statistics based on Scaffold and Size. 𝜃 scaffold and 𝜃 size denote the splitting thresholds used to partition the dataset into in-distribution (DI.D. ) and outof-distribution (DO.O.D. ) subsets.
Stage I: Biomedical Instruction Tuning. In the instruction tuning stage, we aim to ground the LLM with biomedical knowledge relevant to drug and cell contexts. For each training instance, we first retrieve domain-specific biomedical knowledge from curated databases. The retrieved information is organized into a structured biomedical description and utilized by the target response 𝑅. The instruction prompt P𝑖𝑛𝑠𝑡 is then deliberately designed to query, explain, or summarize such biomedical evidence, guiding the language model to generate expert level domain knowledge. During instruction tuning, the large language model is optimized to reproduce the retrieved biomedical descriptions conditioned on the corresponding prompts. Formally, the training objective maximizes the likelihood of generating the target response 𝑅 in the autoregressive token space as follows:
Dataset
𝜃 size (Da)
DI.D.
DO.O.D.
Dtrain
Dvalid
Dtest
Bliss Hsa Loewe Zip
305 305 260 300
1305 878 1753 1295
480 277 294 438
83318 71964 112276 60330
21115 18302 25770 14358
21115 18302 25770 14359
Linst = − log 𝑝 𝑅 | P𝑖𝑛𝑠𝑡 , T𝑐 , TSMILES1 , TSMILES2 ,
(15)
Scaffold-based Splitting Dataset
𝜃 scaffold (mol)
DI.D.
DO.O.D.
Dtrain
Dvalid
Dtest
Bliss Hsa Loewe Zip
13 13 9 13
1364 912 1775 1320
424 243 272 413
84766 72830 109676 59513
20391 17864 27070 14767
20391 17865 27070 14767
Size-based Splitting
4.1
Settings Chemical Space Visualization (t-SNE) — Bliss Dataset
Size Split Random Split Scaffold Split Chemical Space Visualization (t-SNE)— Loewe Dataset
which encourages the model to internalize pharmacological and biological priors through retrieval-augmented supervision. Stage II: Task-Specific Training. In the second stage, we adapt the instruction-tuned DrugSyn-LLM to the DSP task. The prompt P𝑡𝑎𝑠𝑘 is instantiated with task-oriented instructions, while the expected response 𝑅 corresponds to the predicted category and its associated synergy score. In addition to textual inputs, the projected representations E𝑐 , E𝑑1 , and E𝑑2 are injected into the LLM as auxiliary continuous representations. The task training objective is defined in the generative output space of the language model. Specifically, the model is optimized to maximize the likelihood of producing a structured task response 𝑅 = {𝑅label, 𝑅score }, which encodes both the synergy type and the corresponding synergy score as follows: Ltask = − log 𝑝 𝑅 | P𝑡𝑎𝑠𝑘 , T𝑐 , TSMILES1 , TSMILES2 , E𝑐 , E𝑑1 , E𝑑2 .
(16)
Putting All Together. The overall training objective combines the generative task loss with representation and architecture regularization terms as follows: L = 𝛿 Linst + (1 − 𝛿 ) Ltask + 𝛼 Ldecorr + 𝛽 Lsep ,
(17)
where 𝛿 is a stage indicator that activates the instruction-tuning objective, i.e., 𝛿 = 1 during biomedical instruction tuning and 𝛿 = 0 during task-specific training.
4
Experiment
We conduct extensive experiments under two O.O.D. settings for both synergy classification and score regression to evaluate the effectiveness of our proposed OOD-GraphLLM model. Comparisons against a wide range of baselines demonstrate that our method consistently achieves superior performance under distribution shifts.
Scaffold Split
Size Split Test Set
Random Split Train Set
Figure 3: Chemical space visualization.
Dataset. We derive all drug combination data from DrugComb [46], which contains totally 1,432,351 unique <drug, drug, cell line> triplets. Each triplet is annotated with synergy measurements under four scoring schemes, namely Loewe, Bliss, HSA, and ZIP. Detailed definitions and computation rules for these synergy scores are provided in Appendix A. Drug-related information and features is primarily obtained from DrugBank [19], while gene expression profiles of cell lines are collected from the CancerRx-Gene [42] database. We first filter the samples following the recommendation of the SynergyFinder software [13] documentation, retaining only drug combinations that exhibit pronounced synergistic or antagonistic effects (|score| ≥ 10). Subsequently, the filtered dataset is partitioned into DI.D. and DO.O.D. according to the protocol described in Sec 3.1, where scaffold refers to the core chemical framework of a molecule that defines its structural backbone, while size denotes the molecular weight of the compound. Specifically, domains with descriptor values exceeding a predefined threshold 𝜃 are assigned to the training split, while the remaining domains are used for validation and testing, following common practice [16]. Training,
OOD-GraphLLM: Graph Large Language Model for Out-of-Distribution Generalized Drug Synergy Prediction
Conference’17, July 2017, Washington, DC, USA
Table 2: Comparative performance on scaffold-based and size-based O.O.D. DSP classification and regression tasks. The topperforming method is highlighted in bold. We also report the percentage improvement of these metrics compared to the second-best performing method. “-” indicates that the corresponding method does not support this task. Setting Model
Bliss
HSA
Loewe
ZIP
ACC ↑ AUC ↑ MAE ↓ RMSE ↓ ACC ↑ AUC ↑ MAE ↓ RMSE ↓ ACC ↑ AUC ↑ MAE ↓ RMSE ↓ ACC ↑ AUC ↑ MAE ↓ RMSE ↓ Scaffold-Based O.O.D. Drug Synergy Prediction Tasks DeepSynergy 64.53 72.14 24.83 34.05 72.88 75.62 14.19 18.61 93.01 82.28 13.31 17.73 67.16 75.61 17.73 22.32 DFFNDDS 55.98 47.75 66.04 49.58 80.74 53.59 49.91 48.17 TranSynergy 54.95 61.21 26.69 34.41 76.79 73.14 14.51 18.74 92.93 77.66 12.62 17.51 57.46 65.16 17.42 21.17 59.80 62.92 25.56 34.29 77.97 73.71 14.28 18.52 93.11 78.55 12.35 17.33 57.49 67.35 17.07 20.71 DNN- MatchMaker TreeCombo 61.80 68.70 28.08 43.62 72.35 76.32 15.70 19.51 93.05 80.14 14.52 22.90 65.65 75.23 16.24 19.93 Based MarSY 58.51 61.32 27.17 36.07 74.67 73.18 15.42 19.39 92.92 77.22 12.36 17.71 55.19 66.21 17.35 20.89 DTSyn 56.89 59.79 74.42 69.49 63.08 77.67 62.03 69.22 SynergyX 27.76 35.28 15.58 19.34 13.82 19.60 17.51 21.16 DeepDDS 62.10 66.82 68.70 73.95 91.18 74.30 66.61 74.21 DDoS 63.71 70.26 71.16 73.03 91.53 76.38 68.97 76.27 GNN- GAECDS 61.54 61.82 71.23 72.56 91.93 78.94 65.96 71.90 JointSyn 59.07 62.86 26.56 34.41 77.26 69.62 14.73 18.99 92.81 73.56 12.59 18.01 62.36 71.09 16.16 20.33 Based MFSynDCP 65.01 70.60 67.31 73.42 75.21 70.93 68.83 76.07 AttenSyn 55.91 58.51 76.62 70.90 92.76 72.73 58.00 61.15 71.74 81.17 21.74 32.63 77.65 80.66 13.74 18.72 89.52 85.64 13.49 19.30 76.08 78.11 14.21 19.29 LLM- CancerGPT BAITSAO 68.29 75.18 24.77 33.28 75.13 79.85 15.07 18.88 91.61 80.42 14.01 19.18 68.03 76.21 15.02 19.00 Based OOD-GraphLLM 77.27 85.31 20.63 29.24 80.98 83.48 11.74 16.91 93.55 86.19 10.09 15.80 76.98 85.84 11.56 17.17 %↑ +7.71% +5.10% -5.11% -10.39% +3.86% +3.50% -14.56% -8.69% +0.54% +0.64% -25.20% -12.27% +1.18% +9.90% -18.65% -9.63% Size-Based O.O.D. Drug Synergy Prediction Tasks DeepSynergy 68.31 78.05 27.46 35.86 75.85 75.95 14.74 18.57 91.31 78.07 12.99 17.67 73.34 83.12 17.04 24.07 DFFNDDS 54.51 50.95 68.40 50.78 87.46 57.84 51.32 50.67 TranSynergy 59.31 62.45 26.66 35.82 76.48 77.46 14.53 18.43 92.86 76.96 12.19 17.03 61.65 65.17 17.92 22.39 58.18 62.66 26.87 35.58 75.76 71.80 14.39 19.21 92.72 76.54 12.13 17.06 63.98 69.97 17.30 22.02 DNN- MatchMaker TreeCombo 62.54 67.87 27.74 39.09 69.38 73.39 15.83 19.97 92.60 78.35 13.58 19.43 68.22 76.03 16.22 20.53 Based MarSY 56.73 59.26 27.97 36.85 74.11 70.35 14.91 19.53 92.86 77.82 12.13 17.33 53.98 68.49 18.10 22.21 DTSyn 54.76 57.33 73.01 68.03 92.46 77.66 64.28 69.40 SynergyX 26.95 35.98 14.69 19.50 12.48 18.08 18.35 22.56 DeepDDS 65.69 71.29 71.92 77.07 87.98 79.25 68.06 74.09 DDoS 66.90 73.37 72.00 75.37 90.48 72.49 69.06 74.09 GNN- GAECDS 58.54 62.28 71.65 70.23 90.48 79.09 60.87 72.00 59.12 62.40 26.60 34.45 76.08 66.98 15.27 19.78 92.70 79.61 12.23 17.74 63.03 70.14 16.62 21.35 Based JointSyn MFSynDCP 65.28 69.85 65.99 70.61 77.10 71.35 65.94 71.89 AttenSyn 53.86 56.38 72.47 67.60 92.76 78.71 61.93 65.36 CancerGPT 71.92 80.30 23.97 35.39 77.75 78.14 14.00 19.10 93.17 85.23 11.66 17.47 73.99 81.75 14.07 21.65 LLMBAITSAO 69.38 76.91 24.36 33.13 76.59 79.63 14.96 18.79 92.65 80.67 12.65 17.57 69.39 76.40 15.93 21.19 Based OOD-GraphLLM 77.66 85.79 20.85 30.05 79.97 83.00 10.95 16.30 96.17 96.80 10.42 16.00 76.56 85.25 12.66 19.72 %↑ +7.98% +6.84% -13.02% -9.30% +2.86% +4.23% -21.79% -11.56% +3.22% +13.58% -10.63% -6.21% +3.47% +2.56% -10.02% -3.95%
validation, and test sets are organized with an approximate ratio of 4:1:1. Dtailed statistics are reported in Table 1.
Distribution Analysis. Based on a set of generic chemical space descriptors, we visualize the molecular distributions of the Bliss and Loewe datasets under different splitting strategies using t-SNE. As illustrated in Fig. 3, the proposed O.O.D. split yields a clear separation between the train set (DI.D. ) and the test set (DO.O.D. ) across multiple chemical dimensions. This separation substantially increases the difficulty of model generalization, as test compounds reside in chemically distinct regions from those observed during training. In contrast, conventional random splitting, although introducing unseen drugs in the test set, leads to strong overlap and coupling between seen and unseen compounds in chemical space, thereby failing to provide a reliable assessment of a model’s generalization capability.
Baselines. We perform extensive benchmarking against three categories of baseline methods: (i) Conventional DNN-based models: DeepSynergy [31], DFFNDDS [41], TranSynergy [28], MatchMaker [20], TreeCombo [15], MarSY [6], DTSyn [11], and SynergyX [8]; (ii) GNN-based methods: DeepDDS [37], DDoS [32], GAECDS [22], JointSyn [26], MFSynDCP [5], and AttenSyn [38]; (iii) State-of-the-art LLM-based approaches: CancerGPT [25] and BAITSAO [29]. The comparisons across multiple paradigms ensure rigorous evaluations under diverse architectural and learning settings. Drug Descriptors. We retrieve drug-related information from the DrugBank [19] database, including SMILES sequence and basic physicochemical properties for each drug. These drug descriptors are aligned with the downstream drug–drug–cell line triplets using drug names as a common identifier. The SMILES sequences are further processed using the RDKit [21] toolkit to construct graphstructured molecular representations, where atoms and bonds are modeled as nodes and edges. In addition to molecular structures, other retrieved drug attributes are incorporated through carefully designed prompts and corresponding target outputs, which are
Conference’17, July 2017, Washington, DC, USA
ACC(%)
Implementation Details. For classification settings, we evaluate model performance using Accuracy and Area Under the Receiver Operating Characteristic Curve (AUC-ROC). For regression tasks, we adopt Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) as evaluation metrics. We employ galactica [35] pre-trained on large-scale scientific corpora as the backbone LLM architecture. In addition, the cross-modal projection layers are initialized with representations derived from SciBERT [1], providing a semantically informed starting point for multimodal alignment. Most experiments are conducted on NVIDIA A100-SXM4 GPUs with 40 GB memory.
82 80 78 76 74 72
4.2
Results
As shown in Table 2, OOD-GraphLLM consistently outperforms all other baselines across all metrics, datasets, and settings. This demonstrates the superior performance of OOD-GraphLLM in addressing O.O.D. generalized DSP. We have the following key observations: Classification Results. i) The classification metrics on Loewe are consistently higher than those on others. This phenomenon can be attributed to the severe class imbalance in Loewe, under which conditions AUC provides a more reliable measure of discriminative performance. Notably, our method achieves a clear advantage on this metric, highlighting its superior classification capability despite the biased label distribution. ii) LLM-based methods demonstrate superior advantages, suggesting that large language models can leverage semantic information to generalize to O.O.D. drug pairs. By explicitly retrieving and injecting domain-specific medical knowledge, our method achieves a clear performance margin over generic LLM-based approaches. iii) DNN-based and GNN-based methods exhibit no substantial difference in overall performance. Notably, some even underperform DeepSynergy [31], which relies on drug fingerprints, in O.O.D. settings. This suggests that incorporating excessive or highly complex features may introduce significant noise and redundancy, leading models to capture spurious correlations that fail to generalize beyond the training distributions. Regression Results. i) Regression constitutes a more challenging task, as it requires accurate modeling of fine-grained numerical outcomes rather than coarse decision boundaries. Compared with corresponding baselines, our method exhibits a markedly larger performance margin, providing stronger evidence of its effectiveness and accuracy. ii) Several methods are not inherently designed
Scaffold Split - ACC
Scaffold Split - RMSE 19
RMSE
Protein Representations. We obtain the amino acid sequences of target proteins from UniProt [3] and encode them using ESM2 [27]. ESM-2 is a large-scale protein language model pre-trained on millions of protein sequences, which captures rich evolutionary and structural information through self-supervised learning.
82 80 78 76 74 72
ACC(%)
Cell Line Representations. We obtain cell line features from the CancerRx-Gene [42] resource, which provides Robust Multi-array Average (RMA)–normalized [14] basal gene expression profiles for approximately 1000 human cancer cell lines. Each cell line is originally characterized by genome-wide transcriptional measurements covering 17,737 genes. In this study, we focus on a subset of 908 landmark genes curated by the L1000 project [33].
to handle both tasks in a unified manner or are limited to a single prediction paradigm, while others derive classification outcomes indirectly from regression value ranges, which may introduce evaluation bias and compromise result fidelity. Our method is capable of simultaneously generating both outputs in a chain-of-thought manner, enabling coherent reasoning across tasks, which endows OOD-GraphLLM with greater flexibility and scalability.
18 17 16
Size Split - ACC
Size Split - RMSE
19
RMSE
used during the instruction tuning stage in Sec 3.5 to enhance the model’s ability to leverage heterogeneous drug knowledge.
Xin Wang et al.
18 17 16
Full w/o-IT
w/o-Decor w/o-Ctx
w/o-Sep w/o-Att
w/o-NAS
Figure 4: Ablation studies of OOD-GraphLLM.
4.3
Ablation Study
To assess the impact of individual components within our OODGraphLLM, we perform an ablation study on HSA score dataset under both splits. Model performance is evaluated using accuracy for classification and RMSE for regression. We design multiple model variants, where each variant excludes a particular component: • w/o R-IT: We remove the retrieval-augmented biomedical instruction tuning and directly perform task-specific training without external knowledge retrieval. • w/o Decor: We discard the decorrelation constraint Ldecorr , thus removing the enforcement of disentanglement among target-conditioned drug representations. • w/o NAS: We disable the neural architecture search process and instead adopt a handcrafted graph neural network to represent molecular structures. • w/o Attn: We eliminate the pairwise attention mechanism and guide the architecture search using independent drug representations solely. • w/o Sep: We remove the separation constraint Lsep , which enforces the dispersion of operation representations. • w/o Ctx: We exclude structural level cell line features as the contexts and only utilize raw molecular graph topological features. The ablation results are summarized in Fig. 4. Removing any single component consistently leads to noticeable performance degradation across different metrics, indicating that each component contributes meaningfully to our OOD-GraphLLM. Among all variants, w/o Ctx and w/o NAS incur the most pronounced performance drops, particularly on the more challenging regression task. This observation highlights the importance of explicitly modeling cell line contextual information, as well as tailoring neural architectures based on target-adaptive molecular encodings.
OOD-GraphLLM: Graph Large Language Model for Out-of-Distribution Generalized Drug Synergy Prediction
4.4
4.5
Hyperparameter Analysis
To investigate the sensitivity of our framework to key hyperparameters, we analyze the impact of varying 𝛼 and 𝛽 in Eq. 17 on both classification and regression performance in the size-based splitted zip dataset. Specifically, we consider 𝛼, 𝛽 ∈ {0.0, 1e−3, 5e−3, 1e−2, 1e−1}, and report the results in Fig. 5.
75.50 75.25 0.0
1e-3
1e-2
(c) Prompt Design
79 78
30.0
77
29.5
76 75
29.0 P1
P2
Prompt
Scaffold ACC Size ACC
P3 Scaffold RMSE Size RMSE
Mitoxantrone
BRD4
RMSE
21.0
75.0
20.5
74.5
20.0
74.0
Prelamin-A/C
5-Fluorouracil
75.5
0.0
1e-3
5e-3
1e-2
Vorinostat
1e-1
78
31
74
30
72 4
Number of prototypes Scaffold ACC Size ACC
6
Histone Deacetylase 4
Romidepsin
Training
Figure 6: A case study on 5-Fluorouracil and Vorinostat. 32
76
2
Histone deacetylaselike amidohydrolase
Testing
(d) Prototypes per Layer
30.5
RMSE
ACC (%)
5e-3
NCI-H226
21.5
RMSE
75.75
Bliss Score: 10.28
Bliss Score: 10.74
Survival Motor Neuron Protein
ACC RMSE
76.0
Case Study
Thymidylate synthase
(b) Sensitivity to 76.5
ACC (%)
76.00
RMSE
ACC (%)
76.25
21.25 ACC RMSE 21.00 20.75 20.50 20.25 20.00 19.75 1e-1
ACC (%)
(a) Sensitivity to 76.50
Conference’17, July 2017, Washington, DC, USA
29
Scaffold RMSE Size RMSE
Figure 5: Hyperparameter sensitivity analysis for 𝛼, 𝛽, different prompts and prototypes per layer. As shown in the figure, the model achieves the best overall performance when 𝛼 = 5e−3, while increasing or decreasing 𝛼 leads to noticeable performance degradation across different tasks. In contrast, increasing 𝛽 beyond this range has a relatively minor effect on both accuracy and RMSE, whereas reducing 𝛽 results in a more pronounced decline. Consistent with the ablation study, removing either loss term corresponding to 𝛼 or 𝛽 causes a significant drop in performance, highlighting the necessity of both components in our objective. We further evaluate the robustness of our framework to different prompt formulations. We consider three prompts under the same experimental setting on the Bliss dataset: P1: “Do the two drugs exhibit synergy effects? What is their [score] synergy score?”; P2: “Classify the synergy effects between the two drugs and report their [score] synergy score.”; and P3: “As a pharmacovigilance officer, how would you classify and calculate the [score] synergy score between the two drugs?”. The performance remains stable across these prompt variants, suggesting that our framework is not overly sensitive to surface-level prompt wording. This robustness indicates that the model primarily relies on the learned structural, contextual, and semantic representations rather than exploiting a specific prompt template. Finally, we analyze the effect of the number of prototypes per layer in the architecture search module. This hyperparameter controls the capacity of the architecture search space by determining how many prototype operations are maintained at each layer. The results show that increasing the number of prototypes from 2 to 6 consistently improves both classification and regression performance. We use 6 prototypes per layer as the default setting, which achieves the best overall performance in our experiments.
We further present a case study on 5-Fluorouracil and Vorinostat, whose molecular scaffolds and sizes both exhibit substantial topological shifts relative to the training distributions, in the NCI-H226 cell line under Bliss score. Specifically, as shown in Fig. 6, 5-Fluorouracil and Mitoxantrone share a target-conditioned clue related to the survival motor neuron (SMN) protein. Although structurally distinct, both contain polar carbonyl-bearing planar motifs that can support hydrogenbond-mediated interactions around SMN-associated protein-RNA interfaces, suggesting a shared biochemical basis for modulating SMN complex stability rather than acting through direct enzymatic inhibition. This shared polar interaction pattern provides a transferable SMN-related signal for O.O.D. inference. Similarly, Vorinostat and Romidepsin converge on histone deacetylase regulation, particularly HDAC4 and related zinc-dependent HDAC isoforms. Vorinostat inhibits HDACs through hydroxamate-based Zn2+ chelation, whereas Romidepsin, after intracellular reduction, exposes a thiol group that serves an analogous metal-coordinating role. These shared SMN- and HDAC4-centered clues indicate that our targetadaptive modeling can align mechanistically related signals across structurally divergent drugs, thereby supporting generalization under O.O.D. settings.
5
Conclusion
In this work, we propose OOD-GraphLLM, a novel graph LLM framework for out-of-distribution (O.O.D.) generalized drug synergy prediction (DSP) that unifies molecular graph representation and biomedical semantic language representations through a joint optimization. Extensive experiments demonstrate that the proposed OOD-GraphLLM consistently outperforms state-of-the-art approaches on various DSP tasks. To the best of our knowledge, OOD-GraphLLM is the first attempt to study O.O.D. generalized DSP by resorting to graph large language models.
Acknowledgement This work was supported by the National Key Research and Development Program of China No.2023YFF1205001.
References [1] Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. SciBERT: A pretrained language model for scientific text. arXiv preprint arXiv:1903.10676 (2019).
Conference’17, July 2017, Washington, DC, USA
[2] Ziwei Chai, Tianjie Zhang, Liang Wu, Kaiqiang Han, Xiaohai Hu, Xuanwen Huang, and Yang Yang. 2025. Graphllm: Boosting graph reasoning ability of large language model. IEEE Transactions on Big Data (2025). [3] UniProt Consortium. 2019. UniProt: a worldwide hub of protein knowledge. Nucleic acids research 47, D1 (2019), D506–D515. [4] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). 4171–4186. [5] Yunyun Dong, Yunqing Chang, Yuxiang Wang, Qixuan Han, Xiaoyuan Wen, Ziting Yang, Yan Zhang, Yan Qiang, Kun Wu, Xiaole Fan, et al. 2024. MFSynDCP: multi-source feature collaborative interactive learning for drug combination synergy prediction. BMC bioinformatics 25, 1 (2024), 140. [6] Mohamed Reda El Khili, Safyan Aman Memon, and Amin Emad. 2023. MARSY: a multitask deep-learning framework for prediction of drug combination synergy scores. Bioinformatics 39, 4 (2023), btad177. [7] Junfeng Fang, Shuai Zhang, Chang Wu, Zhengyi Yang, Zhiyuan Liu, Sihang Li, Kun Wang, Wenjie Du, and Xiang Wang. 2024. Moltc: Towards molecular relational modeling in language models. arXiv preprint arXiv:2402.03781 (2024). [8] Yue Guo, Haitao Hu, Wenbo Chen, Hao Yin, Jian Wu, Chang-Yu Hsieh, Qiaojun He, and Ji Cao. 2024. SynergyX: a multi-modality mutual attention network for interpretable drug synergy prediction. Briefings in Bioinformatics 25, 2 (2024), bbae015. [9] Betül Güvenç Paltun, Samuel Kaski, and Hiroshi Mamitsuka. 2021. Machine learning approaches for drug combination therapies. Briefings in Bioinformatics 22, 6 (08 2021), bbab293. arXiv:https://academic.oup.com/bib/articlepdf/22/6/bbab293/41088416/bbab293.pdf doi:10.1093/bib/bbab293 [10] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models. ICLR 1, 2 (2022), 3. [11] Jing Hu, Jie Gao, Xiaomin Fang, Zijing Liu, Fan Wang, Weili Huang, Hua Wu, and Guodong Zhao. 2022. DTSyn: a dual-transformer-based neural network to predict synergistic drug combinations. Briefings in Bioinformatics 23, 5 (2022), bbac302. [12] Chao Huang, Xubin Ren, Jiabin Tang, Dawei Yin, and Nitesh Chawla. 2024. Large language models for graphs: Progresses and directions. In Companion Proceedings of the ACM Web Conference 2024. 1284–1287. [13] Aleksandr Ianevski, Anil K Giri, and Tero Aittokallio. 2022. SynergyFinder 3.0: an interactive analysis and consensus interpretation of multi-drug synergies across multiple samples. Nucleic acids research 50, W1 (2022), W739–W743. [14] Francesco Iorio, Theo A Knijnenburg, Daniel J Vis, Graham R Bignell, Michael P Menden, Michael Schubert, Nanne Aben, Emanuel Gonçalves, Syd Barthorpe, Howard Lightfoot, et al. 2016. A landscape of pharmacogenomic interactions in cancer. Cell 166, 3 (2016), 740–754. [15] Joseph D Janizek, Safiye Celik, and Su-In Lee. 2018. Explainable machine learning prediction of synergistic drug combinations for precision cancer medicine. BioRxiv (2018), 331769. [16] Yuanfeng Ji, Lu Zhang, Jiaxiang Wu, Bingzhe Wu, Lanqing Li, Long-Kai Huang, Tingyang Xu, Yu Rong, Jie Ren, Ding Xue, et al. 2023. Drugood: Out-ofdistribution dataset curator and benchmark for ai-aided drug discovery–a focus on affinity prediction problems with noise annotations. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 8023–8031. [17] Bowen Jin, Gang Liu, Chi Han, Meng Jiang, Heng Ji, and Jiawei Han. 2024. Large language models on graphs: A comprehensive survey. IEEE Transactions on Knowledge and Data Engineering (2024). [18] Bowen Jin, Wentao Zhang, Yu Zhang, Yu Meng, Xinyang Zhang, Qi Zhu, and Jiawei Han. 2023. Patton: Language model pretraining on text-rich networks. arXiv preprint arXiv:2305.12268 (2023). [19] Craig Knox, Mike Wilson, Christen M Klinger, Mark Franklin, Eponine Oler, Alex Wilson, Allison Pon, Jordan Cox, Na Eun Chin, Seth A Strawbridge, et al. 2024. DrugBank 6.0: the DrugBank knowledgebase for 2024. Nucleic acids research 52, D1 (2024), D1265–D1275. [20] Halil Ibrahim Kuru, Oznur Tastan, and A Ercument Cicek. 2021. MatchMaker: a deep learning framework for drug synergy prediction. IEEE/ACM transactions on computational biology and bioinformatics 19, 4 (2021), 2334–2344. [21] Greg Landrum. 2013. Rdkit documentation. Release 1, 1-79 (2013), 4. [22] Huijun Li, Lin Zou, Jamal AH Kowah, Dongqiong He, Lisheng Wang, Mingqing Yuan, and Xu Liu. 2023. Predicting drug synergy and discovering new drug combinations based on a graph autoencoder and convolutional neural network. Interdisciplinary Sciences: Computational Life Sciences 15, 2 (2023), 316–330. [23] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning. PMLR, 19730–19742. [24] Lei Li, Hongyu Zhang, Chunhou Zheng, and Yansen Su. 2025. A review of deep learning approaches for drug synergy prediction in cancer. npj Drug Discovery 2, 1 (Dec. 2025), 30. doi:10.1038/s44386-025-00034-1
Xin Wang et al.
[25] Tianhao Li, Sandesh Shetty, Advaith Kamath, Ajay Jaiswal, Xiaoqian Jiang, Ying Ding, and Yejin Kim. 2024. CancerGPT for few shot drug pair synergy prediction using large pretrained language models. NPJ Digital Medicine 7, 1 (2024), 40. [26] Xueliang Li, Bihan Shen, Fangyoumin Feng, Kunshi Li, Zhixuan Tang, Liangxiao Ma, and Hong Li. 2024. Dual-view jointly learning improves personalized drug synergy prediction. Bioinformatics 40, 10 (2024), btae604. [27] Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Salvatore Candido, and Alexander Rives. 2023. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science 379, 6637 (2023), 1123– 1130. arXiv:https://www.science.org/doi/pdf/10.1126/science.ade2574 doi:10. 1126/science.ade2574 [28] Qiao Liu and Lei Xie. 2021. TranSynergy: Mechanism-driven interpretable deep neural network for the synergistic prediction and pathway deconvolution of drug combinations. PLoS computational biology 17, 2 (2021), e1008653. [29] Tianyu Liu, Tinyi Chu, Xiao Luo, and Hongyu Zhao. 2025. Building a unified model for drug synergy analysis powered by large language models. Nature Communications 16, 1 (2025), 4537. [30] Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017). [31] Kristina Preuer, Richard PI Lewis, Sepp Hochreiter, Andreas Bender, Krishna C Bulusu, and Günter Klambauer. 2018. DeepSynergy: predicting anti-cancer drug synergy with deep learning. Bioinformatics 34, 9 (2018), 1538–1546. [32] Kyriakos Schwarz, Alicia Pliego-Mendieta, Amina Mollaysa, Lara Planas-Paz, Chantal Pauli, Ahmed Allam, and Michael Krauthammer. 2022. Ddos: a graph neural network based drug synergy prediction algorithm. arXiv preprint arXiv:2210.00802 (2022). [33] Aravind Subramanian, Rajiv Narayan, Steven M Corsello, David D Peck, Ted E Natoli, Xiaodong Lu, Joshua Gould, John F Davis, Andrew A Tubelli, Jacob K Asiedu, et al. 2017. A next generation connectivity map: L1000 platform and the first 1,000,000 profiles. Cell 171, 6 (2017), 1437–1452. [34] Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. 2024. Graphgpt: Graph instruction tuning for large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 491–500. [35] Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. 2022. Galactica: A large language model for science. arXiv preprint arXiv:2211.09085 (2022). [36] Heng Wang, Shangbin Feng, Tianxing He, Zhaoxuan Tan, Xiaochuang Han, and Yulia Tsvetkov. 2023. Can language models solve graph problems in natural language? Advances in Neural Information Processing Systems 36 (2023), 30840– 30861. [37] Jinxian Wang, Xuejun Liu, Siyuan Shen, Lei Deng, and Hui Liu. 2022. DeepDDS: deep graph neural network with attention mechanism to predict synergistic drug combinations. Briefings in Bioinformatics 23, 1 (2022). [38] Tianshuo Wang, Ruheng Wang, and Leyi Wei. 2023. AttenSyn: an attentionbased deep graph neural network for anticancer synergistic drug combination prediction. Journal of Chemical Information and Modeling 64, 7 (2023), 2854–2862. [39] Xin Wang, Zeyang Zhang, Linxin Xiao, Haibo Chen, Chendi Ge, and Wenwu Zhu. 2025. Towards Multi-modal Graph Large Language Model. arXiv preprint arXiv:2506.09738 (2025). [40] Linxin Xiao, Xin Wang, Zeyang Zhang, Yang Yao, and Wenwu Zhu. 2025. DyNASDDI: Dynamic Pairwise Architecture Search for Generalizable Drug-Drug Interaction LLM. In Proceedings of the 33rd ACM International Conference on Multimedia. 2216–2225. [41] Mengdie Xu, Xinwei Zhao, Jingyu Wang, Wei Feng, Naifeng Wen, Chunyu Wang, Junjie Wang, Yun Liu, and Lingling Zhao. 2023. DFFNDDS: prediction of synergistic drug combinations with dual feature fusion networks. Journal of Cheminformatics 15, 1 (2023), 33. [42] Wanjuan Yang, Jorge Soares, Patricia Greninger, Elena J Edelman, Howard Lightfoot, Simon Forbes, Nidhi Bindal, Dave Beare, James A Smith, I Richard Thompson, et al. 2012. Genomics of Drug Sensitivity in Cancer (GDSC): a resource for therapeutic biomarker discovery in cancer cells. Nucleic acids research 41, D1 (2012), D955–D961. [43] Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu, and Yongfeng Zhang. 2024. Language is all a graph needs. In Findings of the association for computational linguistics: EACL 2024. 1955–1973. [44] Barbara Zdrazil, Eloy Felix, Fiona Hunter, Emma J Manners, James Blackshaw, Sybilla Corbett, Marleen De Veij, Harris Ioannidis, David Mendez Lopez, Juan F Mosquera, et al. 2024. The ChEMBL Database in 2023: a drug discovery platform spanning multiple bioactivity data types and time periods. Nucleic acids research 52, D1 (2024), D1180–D1192. [45] Jianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan, Qian Liu, Rui Li, Xing Xie, and Jian Tang. 2022. Learning on large-scale text-attributed graphs via variational inference. arXiv preprint arXiv:2210.14709 (2022).
OOD-GraphLLM: Graph Large Language Model for Out-of-Distribution Generalized Drug Synergy Prediction
[46] Shuyu Zheng, Jehad Aldahdooh, Tolou Shadbahr, Yinyin Wang, Dalal Aldahdooh, Jie Bao, Wenyu Wang, and Jing Tang. 2021. DrugComb update: a more comprehensive drug sensitivity data repository and analysis portal. Nucleic acids research 49, W1 (2021), W174–W184.
A Experiment Details A.1 Operations To enable flexible architecture search over molecular graph encoders, we define a candidate operator set O (𝑙 ) at each layer 𝑙. All operators are implemented as bond-aware message-passing functions, where node features 𝑥𝑖 and bond features 𝑒𝑖 𝑗 are jointly used to construct molecular messages. • GCNmol. A GCN-style operator that combines neighboring node features with bond features and performs degree-normalized aggregation. • GINmol. A GIN-style operator that applies an MLP to the central node feature and aggregated bond-aware neighbor messages: ∑︁ © ª ℎ𝑖 = MLP (1 + 𝜖 )𝑥𝑖 + ReLU(𝑥 𝑗 + 𝑒𝑖 𝑗 ) ® . 𝑗 ∈N (𝑖 ) « ¬
(18)
• GATmol. An attention-based operator that weights bondaware neighbor messages with attention coefficients 𝛼𝑖 𝑗 : ℎ𝑖 =
∑︁
𝛼𝑖 𝑗 ReLU(𝑥 𝑗 + 𝑒𝑖 𝑗 ).
(19)
𝑗 ∈N (𝑖 )
• SAGEmol. A GraphSAGE-style operator that aggregates bondaware neighborhood information and fuses it with a transformed root-node representation. • Graphmol. A GraphConv-style operator that applies separate transformations to the central node and aggregated neighbor messages before combining them. • MLPmol. A graph-agnostic operator that only applies a learnable linear transformation to the current node feature, serving as a skip-like transformation candidate. All candidate operators share a unified interface. During architecture search, the model learns a weighted combination over these operators, allowing each layer to adaptively select suitable messagepassing functions for molecular representation learning.
A.2
Prompt Design
Prompt:
The cell line of this drug pair is [START_CELL]<DESC>[END_CELL]<Cell Embed> The first drug is [START_SMILES]<SMILES1 >[END_SMILES]<Drug Embed1 > The second drug is [START_SMILES]<SMILES2 >[END_SMILES]<Drug Embed2 >
Instruction Tuning Stage
Q: What are the descriptions of the two drugs?
A: The first drug is <drugname1> and has property <property1>. The second drug is <drug-name2> and has property <property2>.
Task Training Stage
Q: Do the two drugs exhibit synergy effects? What is their <Score_Name> score? A: Yes./No. The absolute value is above <LB> and below <UB>, thus the accurate value is <Score>.
Figure 7: Detailed input prompt design for DrugSyn-LLM. As illustrated in Fig. 7, we carefully design the prompts used to fine-tune DrugSyn-LLM. Specifically, cell line information and drug smiles are incorporated as the backbone prompt to provide
Conference’17, July 2017, Washington, DC, USA
sufficient biological and chemical context. During the instruction tuning stage, the question-answer format is centered on describing and reasoning about intrinsic drug properties. In the subsequent task-specific training stage, we adopt a Chain-of-Thought (CoT) prompting strategy. The model is first guided to predict the categorical synergy outcome, after which it is prompted to infer a bounded interval by specifying the lower and upper bounds of the synergy score. Finally, the model is instructed to output an exact numerical value within this range.
A.3
Instruction Tuning Stategy
To inject reliable biomedical knowledge into the instruction tuning process, we design the retrieval procedure as a curated and deterministic grounding mechanism, which aims to ensure that the language model receives high-quality and consistent textual knowledge during training. Specifically, we construct a local drug-description database from DrugBank [19]. For each drug, we collect its metadata from the IDENTIFICATION section, where the Summary field is used as the primary textual description. For a small number of drugs with missing or incomplete structured entries in DrugBank, we manually verify and supplement their descriptions using trusted biomedical sources such as ChEMBL [44]. During instruction tuning, retrieval is performed by exact matching with DrugBank identifiers. Given a drug in a training instance, its DrugBank ID is used to deterministically map the drug to its corresponding description in the local database. This description is directly formatted into the target instruction text as structured biomedical knowledge.
A.4
Implementation Details
We train the model using the AdamW [30] optimizer with a numerical stability constant of 𝜖 = 1e−8 and apply weight decay (𝜆 = 0.05) as a regularization mechanism. The learning rate is governed by a two-stage schedule, where it is first gradually increased from 1e−6 to 1e−4 during the warm-up phase, and subsequently reduced following a cosine decay strategy until 1e−5. Although different parameter groups are assigned distinct base learning rates, they all share the same global scheduling policy. To enable efficient finetuning of the LLM, we adopt Low-Rank Adaptation (LoRA) [10] with rank 𝑟 = 16, while keeping approximately 99.8% of the original model parameters frozen.
B More Analyses B.1 Message Passing To further understand how information propagates during architecture search, we analyze the operation weights and relate them to molecular structural properties. For each molecule occurrence, we extract the learned operation weights and compute their associations with multiple structural properties. For the correlation between operation weights and structural metrics, we noticed that the operation weights are not uniformly distributed across molecular structures. In particular, GATmol shows a positive Spearman correlation with heavy atom count (𝜌 = 0.4701). Similarly, GRAPHmol is positively correlated with aromatic ring
Conference’17, July 2017, Washington, DC, USA
Xin Wang et al.
Top Attended Target
Mechanistic Relevance
ATM inhibition / DNA damage response Immune activation via 5-Fluorouracil + ImToll-like receptor 7 TLR7 signaling iquimod Oxidative stress and Melphalan hydrochloGlutathione reductase detoxification pathride + Carmustine ways Perifosine + KU-55933
Serine-protein kinase ATM
Substructure-level interpretability. We further analyze whether the model focuses on chemically meaningful molecular substructures. Specifically, we project the final-layer language-model attention scores back to the SMILES characters and identify highattention local fragments. Figure 9 visualizes three representative drugs. Each panel shows the character-level SMILES attention heatmap, with cyan boxes marking the highest-attention fragment. 0.010 0.008
C C C C C C C C C 1 = C C = C ( C = C 1 ) C C C ( C O ) ( C O ) N
0.00
ring_rich
0.05
few_rings moderate
0.10 mol GCN
G
ol INm
mol GAT
ol GEm
SA
Operation
ol PHm GRA
ol LPm
M
Figure 8: Structural-group preferences of diverse messagepassing operations. Colors indicate relative within-group operation preference after subtracting the mean operation weight of each structural group. Positive values indicate operations emphasized within a group, while negative values indicate relatively suppressed operations.
Interpretability
We further examine whether the internal representations of our framework provide biologically and chemically meaningful signals. We conduct interpretability analyses at two levels: target-level attention and substructure-level SMILES attention. Target-level interpretability. We first inspect the cross-attention scores between drug representations and protein-target embeddings. For each analyzed drug-combination sample, we identify the target receiving the highest attention weight and compare it with known pharmacological mechanisms. Representative examples are summarized in Table 3. The attended targets are consistent with established biological knowledge in several cases. For example, KU55933 attends strongly to ATM, which agrees with its known role as an ATM inhibitor involved in DNA damage response. Similarly, Imiquimod attends to Toll-like receptor 7, consistent with its function as a TLR7 agonist.
CCC(CO)(CO)N (C=C1)CCC(CO 1=CC=C(C=C1)
0.0
65.1%
17.5%
0.2
0.4
Decitabine in A375
0.6
0.004
74.4%
0.002
0.8
1.0
0.0
15.0%
0.006
0.2
0.4
0.004
83.1%
31.1%
Nilutamide in A101D
0.6
0.8
0.002
1.0
N + ] ( = O ) [ O - ] ) C ( F ) ( F ) F ) C
0.0
15.0% 0.2
0.000 0.010 0.008
C C 1 ( C ( = O ) N ( C ( = O ) N 1 ) C 2 = C C ( = C ( C = C 2 ) [
])C(F)(F)F)C =O)[O-])C(F) )[N+](=O)[O-
0.000 0.010 0.008
C 1 C ( C ( O C 1 N 2 C = N C ( = N C 2 = O ) N ) C O ) O
NC2=O)N)CO)O N2C=NC(=NC2= (C(OC1N2C=NC
0.006
Attention
0.05
aromatic_rich
Attention
0.10
hetero_rich
B.2
Drug Pair
Fingolimod in IPC-298 Preference
Structural group
large
Table 3: Target-level interpretability examples.
0.004
76.9%
25.2% 0.4
0.6
Fraction of total SMILES attention
0.006
Attention
count (𝜌 = 0.4495), suggesting that graph-level aggregation becomes more important for ring-rich molecular structures. We visualize operation preferences across molecular structural groups in Fig. 8. The visualization reveals clear structural specialization. Small and hetero-atom-rich molecules show stronger preference for GINmol. In contrast, GINmol is suppressed in ring-rich molecules. Ring-rich molecules instead show increased preference for MLPmol, suggesting that the model relies more on feature transformation and less on GINmol-style local expressive aggregation for ring-rich structures. Representative molecule-level examples further support this trend. A small hetero-atom-rich molecule, CN1C(=O)N2C=NC(=C2N =N1)C(=O)N, assigns high weight to GINmol, indicating that expressive local aggregation is useful for compact structures with dense hetero-atom patterns. By contrast, a larger aromatic multi-ring molecule, CS(=O)(=O)C1=CC(=C(C=C1)C(=O)NC2=CC(=C(C=C2)Cl)C3= CC=CC=N3)Cl, shows increased weights for GATmol and GRAPHmol, consistent with the need for attention-based and graph-level propagation over larger aromatic systems.
0.8
0.002
1.0
0.000
Figure 9: Substructure-level SMILES attention visualization for Fingolimod, Decitabine, and Nilutamide. Warmer colors indicate higher character-level attention. Highlighted regions correspond to high-attention local fragments, and the bar plots report their fractions of total SMILES attention. The highlighted fragments are chemically interpretable. For Fingolimod, the model assigns high attention to the amino-alcoholcontaining region, which is consistent with the polar head-group characteristics of this molecule. For Decitabine, the attention concentrates on an N/O-rich nucleoside-like region, matching its chemically distinctive heteroatom-rich structure. For Nilutamide, the model emphasizes the fluorinated aromatic-side fragment, including the trifluoromethyl-related region, which is relevant to lipophilicity and molecular recognition in medicinal chemistry. Overall, these results provide supporting evidence that the learned representations capture signals aligned with known pharmacological and medicinal-chemistry knowledge.