Learning relevant contextual variables within Bayesian optimization

Published: 27 Oct 2023, Last Modified: 22 Dec 2023RealML-2023EveryoneRevisionsBibTeX
Keywords: Contextual Bayesian Optimization, Variable Selection, Gaussian Processes
TL;DR: We propose a variable selection method for identifying relevant contextual variables in Bayesian Optimization
Abstract: Contextual Bayesian Optimization (CBO) efficiently optimizes black-box, expensive-to- evaluate functions with respect to design variables, while simultaneously integrating relevant contextual information regarding the environment, such as experimental conditions. However, the relevance of contextual variables is not necessarily known beforehand. Moreover, contextual variables can sometimes be optimized themselves, an overlooked setting by current CBO algorithms. Optimizing contextual variables may be costly, which raises the question of determining a minimal relevant subset. We address this problem using a novel method, Sensitivity-Analysis-Driven Contextual BO (SADCBO). We learn the relevance of context variables by sensitivity analysis of the posterior surrogate model, whilst minimizing the cost of optimization by leveraging recent developments on early stopping for BO. We empirically evaluate our proposed SADCBO against alternatives on both synthetic and real-world experiments, and demonstrate a consistent improvement across examples.
Submission Number: 59