De Novo Drug Design with Joint Transformers

Adam Izdebski, Ewelina Weglarz-Tomczak, Ewa Szczurek, Jakub Tomczak

Keywords: transformers, generative ai, drug design

TL;DR: We propose Joint Transformer with a probabilistic black-box optimization algorithm that achieves SOTA results on three GuacaMol benchmarks.

Abstract: De novo drug design requires simultaneously generating novel molecules outside of training data and predicting their target properties, making it a hard task for generative models. To address this, we propose Joint Transformer that combines a Transformer decoder, Transformer encoder, and a predictor in a joint generative model with shared weights. We show that training the model with a penalized log-likelihood objective results in state-of-the-art performance in molecule generation, while decreasing the prediction error on newly sampled molecules, as compared to a fine-tuned decoder-only Transformer, by 42%. Finally, we propose a probabilistic black-box optimization algorithm that employs Joint Transformer to generate novel molecules with improved target properties and outperform other SMILES-based optimization methods in de novo drug design.

Submission Number: 93