Accelerating Model-Free Optimization via Averaging of Cost Samples

Guido Carnevale, Giuseppe Notarstefano

Keywords: Optimization, Model-Free Optimization, Zeroth-Order Optimization, Derivative-Free Optimization

TL;DR: Model-free optimization methods typically use only current cost samples (e.g., one per iteration) by discarding all the past cost samples. We introduce a simple yet memory mechanism to maintain and use them until new data become available.

Abstract: Model-free optimization methods typically rely on cost samples gathered by perturbing the current solution estimate along a finite and fixed set of directions. However, at each iteration, only the current cost samples are used, while potentially informative, previously collected samples are discarded. In this work, we challenge this conventional approach by introducing a simple yet effective memory mechanism that maintains an auxiliary vector of iteratively updated cost samples. By leveraging this stored information, our method estimates descent directions through an averaging of all perturbing directions weighted by the auxiliary vector components. This results in a faster convergence without increasing the number of function queries. By interpreting the resulting algorithm as a time-varying dynamical system, we are able to establish its convergence properties in the strongly convex case. In particular, by using tools from system theory based on timescale separation, we are able to guarantee a linear convergence rate toward an arbitrarily small neighborhood of the optimal solution. Numerical simulations on regression problems demonstrate that the proposed approach significantly outperforms existing model-free optimization methods.

Primary Area: Optimization (e.g., convex and non-convex, stochastic, robust)

Submission Number: 12932