Local Entropy Search over Descent Sequences for Bayesian Optimization

ICLR 2026 Conference Submission17977 Authors

Keywords: Bayesian Optimization, Entropy Search, Local Bayesian Optimization

TL;DR: Local Entropy Search is a Bayesian optimization algorithm that reduces uncertainty over an optimizer’s descent path, enabling sample-efficient local search for high complexity problems.

Abstract: Searching large and highly complex design spaces for a global optimum can be infeasible and unnecessary. A practical alternative is to iteratively refine the neighborhood of an initial design using local optimization methods such as gradient descent. We propose local entropy search (LES), a Bayesian optimization paradigm that explicitly targets the solutions reachable by the descent sequences of arbitrary iterative optimizers. The algorithm propagates the posterior belief over the objective through the optimizer, yielding a probability distribution over descent sequences. It then selects the next evaluation by maximizing mutual information with that distribution, using a practical combination of analytic entropy calculations and Monte-Carlo sampling of descent sequences. Empirical results on high-complexity synthetic objectives and benchmark problems show that LES achieves strong sample efficiency compared to existing local and global Bayesian optimization methods.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 17977