Transfer Learning of Condition-Specific Perturbation in Gene Interactions: Towards Multi-modal Foundational Modeling of Drug Response

ICML 2025 Workshop FM4LS Submission17 Authors

Keywords: Transfer learning, Transcriptome, Drug response, Gene Attention

TL;DR: We introduce CSG$^2$A, a multi-modal attention-based framework that integrates chemical and transcriptomic data to model drug response across biological scales from gene to cell and potentially patients.

Abstract:

Effective modeling of drug response requires a multi-scale framework that bridges molecular perturbations with cellular viability. While emerging biological foundation models offer promise for cross-scale transfer, they are not explicitly designed to capture chemical-induced perturbations across omics and chemical modalities. In this work, we present Condition-Specific Gene-Gene Attention (CSG$^2$A), a multi-modal transfer learning framework that integrates transcriptomic profiles with compound structure and treatment conditions (e.g., dosage, time) to model drug responses at multiple scales.
CSG$^2$A is first pretrained on large-scale drug-induced gene expression perturbation dataset to learn condition-aware gene interaction patterns through its gene-gene attention module, guided by interactome network priors. It is then transferred to cell viability dataset, achieving state-of-the-art performance in cell line drug response prediction. Case studies support the biological interpretability of the learned attention maps, aligning with known drug mechanisms. CSG$^2$A also generalizes to patient-level prediction on TCGA, demonstrating its potentials in cross-scale transfer and offering promising directions for developing multi-modal foundation models in drug response.

Submission Number: 17