Blackbox optimization of unimodal functions

Ashok Cutkosky, Abhimanyu Das, Weihao Kong, Chansoo Lee, Rajat Sen

Keywords: unimodal function, black box optimization, automl

TL;DR: Simple but optimal method for black-box optimization of unimodal functions with practical application to hyper-parameter tuning

Abstract: We provide an intuitive new algorithm for blackbox stochastic optimization of unimodal functions, a function class that we observe empirically can capture hyperparameter-tuning loss surfaces. Our method's convergence guarantee automatically adapts to Lipschitz constants and other problem difficulty parameters, recovering and extending prior results. We complement our theoretical development with experimental validation on hyperparameter tuning tasks.