Active and transfer learning with partially Bayesian neural networks for materials and chemicals

Sarah I. Allec, Maxim Ziatdinov

Submission Track: Full Paper

Submission Category: AI-Guided Design

Keywords: active learning, Bayesian neural networks, uncertainty quantification, materials informatics, cheminformatics

TL;DR: We demonstrate the potential of partially Bayesian neural networks for active and transfer learning for limited, complex materials and chemical datasets.

Abstract:

Active learning, an iterative process of selecting the most informative data points
for exploration, is crucial for efficient characterization of materials and chemicals
property space. Neural networks excel at predicting these properties but
lack the uncertainty quantification needed for active learning-driven exploration.
Fully Bayesian neural networks, in which weights are treated as probability distributions
inferred via advanced Markov Chain Monte Carlo methods, offer robust
uncertainty quantification but at high computational cost. Here, we show
that partially Bayesian neural networks (PBNNs), where only selected layers have
probabilistic weights while others remain deterministic, can achieve accuracy and
uncertainty estimates on active learning tasks comparable to fully Bayesian networks
at lower computational cost. Furthermore, by initializing prior distributions
with weights pre-trained on theoretical calculations, we demonstrate that PBNNs
can effectively leverage computational predictions to accelerate active learning
of experimental data. We validate these approaches on both molecular property
prediction and materials science tasks, establishing PBNNs as a practical tool for
active learning with limited, complex datasets.

Submission Number: 50