An Adaptive Parallel Gaussian Homotopy Approach for Zeroth-Order Optimization

Bingyi Zhou, Guoyou Wang

Abstract: The Gaussian homotopy method is a classical optimization approach for solving nonconvex problems. It applies Gaussian smoothing to a given problem, using varying smoothing factors to generate a series of proxy problems that are easier to solve, subsequently addressing them gradually from simpler to more complex ones. Traditional Gaussian homotopy methods typically utilize a predefined sequence or parameter to update a single smoothing factor serially. However, this sequential updating process is inefficient and unsuitable for parallel homotopy pipelines. Moreover, the utilization of predefined sequences or parameters leads to a lack of adaptability. To address the abovementioned challenges, we propose an adaptive parallel Gaussian homotopy optimization method. Initially, we introduce an adaptive model called Scaling. This model is formulated as a scaler that concurrently adjusts multiple smoothing factors, with each factor aligning to a homotopy level. It can be integrated easily into serial and parallel homotopy pipelines. Furthermore, we establish a collaborative training regimen to jointly train the Scaling model and a parallel homotopy model, named continuation path learning (CPL) [1] model. Throughout the training process, the Scaling model furnishes CPL with multiple scaled smoothing factors and updates CPL implicitly. Extensive experiments demonstrate that the proposed Gaussian homotopy approach performs competitively.