Learning to [locally] optimize a black box

Semenov Maxim Dmitrievich, Grigori Nikitin, Nikolai Kovrizhnykh, Feodor Pisnitchenko

Keywords: black-box optimization, learning to optimize, meta-learning, machine learning, optimization methods, data-driven optimization

TL;DR: Leveraging machine learning to develop efficient black-box optimization methods for costly evaluations.

Abstract:

We consider black-box optimization with a limited number of function evaluations.
This is a typical scenario when optimizing variable settings that are expensive to evaluate.

The traditional designs of optimization methods are theory-driven, so they obtain performance guarantees over the classes of problems specified by the theory.
In sharp contrast, the concept of Learning to Optimize (L2O) represents a novel strategy that employs machine learning techniques to create optimization algorithms.
This approach automates the design of an optimization algorithm for a class of optimization problems.
This data-driven procedure generates methods that can efficiently solve problems similar to those in the training set.

Submission Number: 74