Lipschitz Optimization via Weighted Sampling Based on Expected Potential Maximizers Reduction

Hideyuki Masui, Koki Nakane, Renshi Nagasawa

Keywords: Lipschitz Optimization, Bayesian Optimization, Black-box Optimization, Gaussian Process, Acquisition Function

Abstract: We address the problem of black-box optimization under the assumption that the objective function is Lipschitz-continuous. In traditional Lipschitz optimization algorithms, potential maximizers are defined based on lower and upper bounds estimated from observed data, and new query points are selected uniformly at random from this set. In contrast, we propose a weighted sampling strategy guided by a probabilistic model, where each candidate point is assigned a weight corresponding to its expected reduction of the potential maximizers. This enables more informative and efficient exploration. To retain theoretical guarantees, we incorporate a small probability of uniform sampling from the potential maximizers, ensuring convergence to the global optimum. We demonstrate the effectiveness of the proposed method through numerical experiments on standard benchmark functions and hyperparameter optimization tasks using UCI datasets.

Submission Number: 31