Graph Diffusion Transformers are In-Context Molecular Designers

ICLR 2026 Conference Submission19849 Authors

Keywords: Inverse Molecular Design, In Context Learing, Diffusin Models, Transformers

Abstract: In-context learning lets large models adapt to new tasks from a few demonstrations, but it has shown limited success in molecular design, where labeled data are scarce and properties span millions of biological assays and material measurements. We introduce demonstration-conditioned diffusion models (DemoDiff), which define task contexts through molecule–score examples instead of texts. These demonstrations guide a denoising Transformer to generate molecules aligned with target properties. For scalable pretraining, we develop a new molecular tokenizer with Node Pair Encoding that represents molecules at the motif level, requiring 5.5$\times$ fewer nodes. We pretrain a 0.7B parameter model on datasets covering drugs and materials. Across 33 design tasks in six categories, DemoDiff matches or surpasses language models 100–1000$\times$ larger and achieves an average rank of 3.63 compared to 5.25–10.20 for domain-specific approaches. These results position DemoDiff as a molecular foundation model for in-context molecular design.

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Submission Number: 19849