Matbench Discovery - Can machine learning identify stable crystals?

Janosh Riebesell, Rhys Goodall, Anubhav Jain, Kristin Persson, Alpha Lee

Keywords: materials discovery, inorganic crystal stability, machine learning, interatomic potential, Bayesian optimization, high-throughput search, convex hull, formation energy

TL;DR: We benchmark ML models on crystal stability prediction from unrelaxed structures finding interatomic potentials in particular to be a valuable addition to high-throughput discovery pipelines.

Abstract:

We present a new machine learning (ML) benchmark for thermodynamic materials stability predictions named \texttt{Matbench Discovery}.
A goal of this benchmark is to highlight the need to focus on metrics that directly measure their utility in prospective discovery campaigns as opposed to analyzing models based on predictive accuracy alone.
Our benchmark consists of a task designed to closely simulate the deployment of ML energy models in a high-throughput search for stable inorganic crystals.
We explore a wide variety of models covering multiple methodologies ranging from random forests to GNNs, and from one-shot predictors to iterative Bayesian optimizers and interatomic potential-based relaxers. We find M3GNet to achieve the highest F1 score of 0.58 and $R^2$ of 0.59 while MEGNet wins on discovery acceleration factor (DAF) with 2.70. Our results provide valuable insights for maintainers of high throughput materials databases to start using these models as triaging steps to more effectively allocate compute for DFT relaxations.