Enhancing generative perturbation models with LLM-informed gene embeddings

Kaspar Märtens, Rory Donovan-Maiye, Jesper Ferkinghoff-Borg

Keywords: perturbations; generative models; LLMs

TL;DR: We incorporate prior knowledge in the form of LLM-based gene embeddings to improve generative perturbation models

Abstract: Genetic perturbations are key to understanding how genes regulate cell behavior, yet the ability to predict responses to these perturbations remains a significant challenge. While numerous generative models have been developed for perturbation data, they typically lack the capability to generalize to perturbations not encountered during training. To alleviate this limitation, we introduce a novel methodology that incorporates prior knowledge through embeddings derived from Large Language Models (LLMs), effectively informing our predictive models with a deeper biological context. By leveraging this source of pre-existing information, our models achieve state-of-the-art performance in predicting the outcomes of single-gene perturbations.

Submission Number: 63