Exploring Transfer Learning for Materials Property Prediction

Bashar Albakri, Alexander Kister, Phlipp Benner, Janine george

Keywords: Transfer Learning, Matbench, MLIP, Materials Foundation Models, Negative Transfer

Abstract: Transfer learning based on foundation models has shown strong gains in low-data regimes, and similar benefits are emerging for materials property prediction where labeled data are scarce due to the cost of DFT calculations and experimental measurements. We study transfer learning for materials properties by finetuning MACE-MP-0, a relatively small foundation model pretrained on large-scale structure–energy DFT data, on different tasks from Matbench spanning mechanical and electronic targets. Energy-based pretraining yields consistent improvements over random initialization. We further show that negative transfer can limit gains and that regularization can improve results. Additionaly we provide some interpretations on what is transferred via layer-wise mutual information and weight-space drift.

Submission Track: Paper Track (Tiny Paper)

Submission Category: Automated Material Characterization

Submission Number: 28