Small Molecule Optimization with Large Language Models
Keywords: Large Language Models, Molecule Generation, Transfer Learning, Drug Discovery, Evolutionary Algorithms
Abstract: Recent advancements in large language models (LLMs) have opened new possibilities for generative molecular drug design. In molecular optimization, LLMs are promising candidates to augment traditional modeling and rule-based approaches for refining molecular structures toward design criteria. We present a novel approach to molecular optimization using LLMs trained on a hand-crafted corpus of over 100 million molecules and their properties. We trained three new models, Chemlactica-125M, Chemlactica-1.3B, and Chemma-2B, with a demonstrated ability to generate molecules with specified properties and learn new molecular characteristics from limited samples, competitive with the state-of-the-art (SOTA) in property prediction tasks on experimental data. Our optimization method, elucidated by these capabilities, combines the models' generative power with concepts from prompt optimization, evolutionary algorithms, and rejection sampling to solve molecular optimization problems more efficiently. The approach surpasses previous SOTA results on the Practical Molecular Optimization (PMO) benchmark and exceeds or is competitive with the SOTA in multi-property optimization tasks involving docking simulations. We release the training data, language models, and optimization algorithm to facilitate further research and reproducibility.
Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 10806
Loading