OrthoBO: Orthogonalized Bayesian Hyperparameter Optimization

Maresa Schröder, Pascal Janetzky, Michael Klar, Stefan Feuerriegel

Keywords: Bayesian optimization, hyperparameter optimization

Abstract: Bayesian optimization is widely used for hyperparameter optimization when model evaluations are expensive, but noisy acquisition estimates can lead to unstable decisions. We identify acquisition estimation noise as a distinct failure mode: even when the surrogate model and acquisition target are appropriate, finite-sample Monte Carlo error can perturb acquisition values, flip candidate rankings, and lead Bayesian optimization to evaluate suboptimal configurations. As a remedy, we propose an *orthogonal acquisition estimator*, that subtracts an optimally weighted score-function control variate, yielding an acquisition residual orthogonal to posterior score directions and thereby reducing Monte Carlo variance. Building on this estimator, we introduce OrthoBO, a Bayesian optimization framework that combines orthogonalized acquisition estimates with ensemble surrogates for structural misspecification and an outer log transformation for numerical stability. Theoretically, we prove target preservation, variance reduction, and improved pairwise ranking stability. Empirically, OrthoBO substantially reduces acquisition estimation variance, stabilizes candidate rankings, and achieves strong performance across synthetic benchmarks and downstream use cases, including vision-transformer fine-tuning on an industrial wafer-map classification task. These results show that stabilizing acquisition estimation can directly improve the reliability and sample efficiency of Bayesian hyperparameter optimization.

Submission Number: 48