Applications of Modular Co-Design for De Novo 3D Molecule Generation

Danny Reidenbach, Filipp Nikitin, Olexandr Isayev, Saee Gopal Paliwal

Keywords: molecule generation, diffusion, flow matching, transformer

TL;DR: Megalodon, a simple and scalable transformer model, produces more valid 3D molecules with low structure error and lower-energy molecules for drug discovery.

Abstract:

De novo 3D molecule generation is a pivotal task in drug discovery. However, many
recent geometric generative models struggle to produce high-quality 3D structures,
even if they maintain 2D validity and topological stability. To tackle this issue
and enhance the learning of effective molecular generation dynamics, we present
Megalodon–a family of simple and scalable transformer models. These models
are enhanced with basic equivariant layers and trained using a joint continuous
and discrete denoising co-design objective. We assess Megalodon’s performance
on established molecule generation benchmarks and introduce new 3D structure
benchmarks that evaluate a model’s capability to generate realistic molecular
structures, particularly focusing on energetics. We show that Megalodon achieves
state-of-the-art results in 3D molecule generation, conditional structure generation,
and structure energy benchmarks using diffusion and flow matching. Furthermore,
we demonstrate that scaling Megalodon produces up to 49x more valid molecules
at large sizes and 2-10x lower energy compared to the prior best generative models.

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2898