Pre-training Transformers for Molecular Property Prediction Using Reaction PredictionDownload PDF

Published: 15 Jun 2022, Last Modified: 05 May 2023ICML-AI4Science PosterReaders: Everyone
Keywords: molecular property prediction, pre-training, transfer learning, transformers, cheminformatics
TL;DR: Exploring reaction prediction as a pre-training step for transformers on downstream molecular property prediction tasks
Abstract: Molecular property prediction is essential in chemistry, especially for drug discovery applications. However, available molecular property data is often limited, encouraging the transfer of information from related data. Transfer learning has had a tremendous impact in fields like Computer Vision and Natural Language Processing signalling for its potential in molecular property prediction. We present a pre-training procedure for molecular representation learning using reaction data and use it to pre-train a SMILES Transformer. We fine-tune and evaluate the pre-trained model on 12 molecular property prediction tasks from MoleculeNet within physical chemistry, biophysics, and physiology and show a statistically significant positive effect on 5 of the 12 tasks compared to a non-pre-trained baseline model.
Track: Original Research Track
0 Replies