Multi-modal Foundation Model for Material Design

Seiji Takeda, Indra Priyadarsini, Akihiro Kishimoto, Hajime Shinohara, Lisa Hamada, Hirose Masataka, Junta Fuchiwaki, Daiju Nakano

Submission Track: Papers

Submission Category: AI-Guided Design

Keywords: foundation model, muti modality, deep learning, generative model, property prediction

TL;DR: We introduce a multi-modal foundation model for small molecules, pre-trained with over 6 billion sparse data samples including SELFIES, DFT properties, and optical absorption spectrum.

Abstract: We propose a multi-modal foundation model for small molecules, a shift from traditional AI models that are tailored for individual tasks and modalities. This model uses a late fusion strategy to align and fuse three distinct modalities: SELFIES, DFT properties, and optical spectrum. The model is pre-trained with over 6 billion samples to provide two primary functions, generating fused feature representations across the three modalities, and cross-modal predictions and genrations. As preliminary experiments, we demonstrate that the fused representation successfully improves the performance of property predictions for chromophore molecules, and showcase 6 distinct cross-modal inferences.

Digital Discovery Special Issue: Yes

Submission Number: 20