Boltz-1 as a force field -- why co-folding models struggle with learning physics and how to fix it

Urszula Julia Komorowska, Vsevolod Viliuga, Leif Seute, Pietro Lio, Mateja Jamnik

Keywords: structure prediction, machine learning interatomic potentials, protein design, co-folding

Abstract: Modern co-folding models have achieved remarkable success in biomolecular structure prediction. However, their ability to generalise to \mbox{out-of-distribution} examples and to capture physical laws remains limited. These limitations may stem from either data or architecture; here, we focus on the latter by examining whether the training objectives and architectural choices of co-folding models hinder the learning of physical laws. Drawing on insights from physics-constrained Machine Learning Interatomic Potentials (MLIPs), we investigate the expressiveness of attention-based modules as implemented in the co-folding models. We evaluate the exemplary co-folding model \mbox{Boltz-1} as an MLIP and find that it underperforms on energy surface learning. Our analysis shows that accurate energy learning requires inter-atomic distances to be encoded appropriately in the attention pair bias, whereas \mbox{Boltz-1} constructs these features in a way that fails to support this task. We further identify \mbox{Boltz-1}’s strong reliance on pairformer features as an additional limitation in this context, even though this reliance might be beneficial in the structure prediction task. Based on these insights from MLIPs, we introduce simple architectural modifications, including a revised pair bias encoding, and show that they significantly improve energy landscape learning.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 37