HiBBO: HiPPO-based Space Consistency for High-dimensional Bayesian Optimisation

Junyu Xuan, Wenlong Chen, Yingzhen Li

Keywords: HiPPO, Bayesian Optimisation, VAE

TL;DR: A novel BO framework that introduces the space consistency into the latent space construction in VAE using HiPPO

Abstract: Bayesian Optimisation (BO) is a powerful tool for optimising expensive black-box functions, but its effectiveness diminishes in high-dimensional spaces due to sparse data and poor surrogate model scalability. While Variational Autoencoder (VAE)-based approaches address this by learning low-dimensional latent representations, the reconstruction-based objective function often brings the functional distribution mismatch between the latent space and original space, leading to suboptimal optimisation performance. In this paper, we first analyse the reason why reconstruction-only loss may lead to distribution mismatch and then propose HiBBO, a novel BO framework that introduces the space consistency into the latent space construction in VAE using HiPPO—a method for long-term sequence modelling—to reduce the functional distribution mismatch between the latent space and original space. Experiments on high-dimensional benchmark tasks demonstrate that HiBBO outperforms existing VAE-BO methods in convergence speed and solution quality. Our work bridges the gap between high-dimensional sequence representation learning and efficient Bayesian Optimisation, enabling broader applications in neural architecture search, materials science, and beyond.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 5628