Why risk matters for protein binder design

Tudor-Stefan Cotet, Igor Krawczuk

Track: Machine learning: computational method and/or computational results

Nature Biotechnology: Yes

Keywords: Bayesian Optimization, Protein Engineering, Risk Analysis, Model Selection, Benchmarking, Active Learning, Fitness Landscapes, Experimental Design, Portfolio Optimization, Protein-Protein Binding, Uncertainty Quantification, risk-aware optimization, risk-aware benchmarking

TL;DR: We benchmark 72 BO variants, quantifying risk, costs, and performance trade-offs in model selection across 11 deep mutational scanning datasets measuring risk and showing it gives landscape specific information beyond the average performance.

Abstract: Bayesian optimization (BO) has recently become more prevalent in protein engineering applications and hence has become a fruitful target of benchmarks. However, current BO comparisons often overlook real-world considerations like risk and cost constraints. In this work, we compare 72 model combinations of encodings, surrogate models, and acquisition functions on 11 protein binder fitness landscapes, specifically from this perspective. Drawing from the portfolio optimization literature, we adopt metrics to quantify the cold-start performance relative to a random baseline, to assess the risk of an optimization campaign, and to calculate the overall budget required to reach a fitness threshold. Our results suggest the existence of Pareto-optimal models on the risk-performance axis, the shift of this preference depending on the landscape explored, and the robust correlation between landscape properties such as epistasis with the average and worst-case model performance. They also highlight that rigorous model selection requires substantial computational and statistical efforts.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Presenter: ~Tudor-Stefan_Cotet1

Format: Yes, the presenting author will attend in person if this work is accepted to the workshop.

Funding: No, the presenting author of this submission does *not* fall under ICLR’s funding aims, or has sufficient alternate funding.

Submission Number: 38