Crystal Design Amidst Noisy DFT Signals: A Reinforcement Learning Approach

Prashant Govindarajan, Mathieu Reymond, Santiago Miret, Mariano Phielipp, Sarath Chandar

Submission Track: Findings & Open Challenges

Submission Category: AI-Guided Design

Keywords: Reinforcement learning, DFT, band gap, online learning

TL;DR: Incorporating DFT simulation in active learning inspired online RL experiments for band gap directed crystal design.

Abstract: In-silico design of novel materials demands a large number of atom-level calculations for optimizing the desired properties. In practice, it is extremely time-consuming and cumbersome to perform density functional theory calculations at an exponential scale. In the hope of accelerating material discovery, we investigate the feasibility of an active learning-inspired reinforcement learning approach based on online reward model fine-tuning to learn a policy that can generate compositions of crystalline materials optimized for a specific band gap. Through an extensive set of online learning experiments, we show that while RL policies can be effectively trained using machine learning-based proxy reward functions, they fail to converge for DFT-based rewards. This failure of convergence could be related to the inherently noisy nature of DFT in resolving the electronic band structure, which severely affects policy learning. To this end, we emphasize the need for more specialized and domain-driven methods for band gap optimization.

AI4Mat Journal Track: Yes

Submission Number: 84