SupSiam: Non-contrastive Auxiliary Loss for Learning from Molecular Conformers

Michael Maser, Ji Won Park, Joshua Yao-Yu Lin, Jae Hyeon Lee, Nathan C. Frey, Andrew Martin Watkins

Keywords: equivariance, representation learning, conformer ensembles, activity prediction, Siamese networks, augmentation

Abstract: We investigate Siamese networks for learning related embeddings for augmented samples of molecular conformers. We find that a non-contrastive (positive-pair only) auxiliary task aids in supervised training of Euclidean neural networks (E3NNs) and increases manifold smoothness (MS) around point-cloud geometries. We demonstrate this property for multiple drug-activity prediction tasks while maintaining relevant performance metrics, and propose an extension of MS to probabilistic and regression settings. We provide an analysis of representation collapse, finding substantial effects of task-weighting, latent dimension, and regularization. We expect the presented protocol to aid in the development of reliable E3NNs from molecular conformers, even for small-data drug discovery programs.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/supsiam-non-contrastive-auxiliary-loss-for/code)