Applications of Modular Co-Design for De Novo 3D Molecule Generation

Danny Reidenbach, Filipp Nikitin, Olexandr Isayev, Saee Gopal Paliwal

Keywords: diffusion, model scaling, molecule generation, flow matching

TL;DR: Megalodon, a simple and scalable transformer model, produces more valid 3D molecules with low structure error and lower-energy molecules for drug discovery.

Abstract: De novo 3D molecule generation is a pivotal task in drug discovery. However, many recent geometric generative models struggle to produce high-quality 3D structures, even if they maintain 2D validity and topological stability. To tackle this issue and enhance the learning of effective molecular generation dynamics, we present Megalodon–a family of simple and scalable transformer models. These models are enhanced with basic equivariant layers and trained using a joint continuous and discrete denoising co-design objective. We assess Megalodon’s performance on established molecule generation benchmarks and introduce new 3D structure benchmarks that evaluate a model’s capability to generate realistic molecular structures, particularly focusing on energetics. We show that Megalodon achieves state-of-the-art results in 3D molecule generation, conditional structure generation, and structure energy benchmarks using diffusion and flow matching. Furthermore, we demonstrate that scaling Megalodon produces up to 49x more valid molecules at large sizes and 2-10x lower energy compared to the prior best generative models.

Submission Number: 18