How to Spend Your Oracle Budget: Practical Guidance for Protein Structure Foundation Models

Aleksandra Kalisz, Jack Simons, Krisztina Sinkovics, Noam Ghenassia, Shikha Surana, Henry Moss, Paul Duckworth

Keywords: Generative Models, Latent Space Optimisation, Bayesian Optimisation, Structure Prediction, Proteins, AI4Science

TL;DR: We compare popular guidance methods for protein structure prediction models in budget-constrained scenarios, along with the first implementation of O3 (optimisation over outputs) for Boltz-2 model.

Abstract: Foundation models for protein structure prediction remain unreliable on certain targets. External oracles can flag and correct these failures, but biological oracles are expensive, making oracle budget a critical constraint. Existing guidance methods, such as FK-steering, DPO, and Best K-of-N sampling, differ in how they spend this budget, yet no systematic comparison exists to guide method selection. To bridge this gap, we benchmark these methods alongside the recently proposed Optimisation Over Outputs (O3), which applies off-the-shelf optimisers within a generative model's latent subspace. We extend the usage of O3 to protein structure models. Overall our work provides the first practical reference for oracle budget-aware guidance. Our evaluation on the calmodulin target 1CLL reveals that no single method consistently dominates across all budgets. Specifically, O3 proves most effective at low-to-mid oracle budgets, while FK-steering and DPO demonstrate superior performance as the budget increases. We distil these findings into actionable recommendations for practitioners operating under real-world oracle-budget constraints.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 62