Regime-Adaptive Bayesian Optimization via Dirichlet Process Mixtures of Gaussian Processes

Chong Liu, Sascha Ranftl, Xuefeng Liu, Shibo Li

Abstract:

Standard Bayesian Optimization (BO) assumes uniform smoothness across the search
space—an assumption violated in multi-regime problems such as molecular conformation
search through distinct energy basins or drug discovery across heterogeneous molecu-
lar scaffolds. A single GP either oversmooths sharp transitions or hallucinates noise in
smooth regions, yielding miscalibrated uncertainty. We propose RAMBO, a Dirichlet
Process Mixture of Gaussian Processes that automatically discovers latent regimes during
optimization, each modeled by an independent GP with locally-optimized hyperparame-
ters. We derive collapsed Gibbs sampling that analytically marginalizes latent functions
for efficient inference, and introduce adaptive concentration parameter scheduling for
coarse-to-fine regime discovery. Our acquisition functions decompose uncertainty into
intra-regime and inter-regime components. Experiments on synthetic benchmarks and
real-world applications—including molecular conformer optimization, virtual screening for
drug discovery, and fusion reactor design—demonstrate consistent improvements over state-
of-the-art baselines on multi-regime objectives. Our implementation is publicly available at
https://github.com/AnthonyZhangYan/RAMBO