Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges

Bernd Bischl, Martin Binder, Michel Lang, Tobias Pielok, Jakob Richter, Stefan Coors, Janek Thomas, Theresa Ullmann, Marc Becker, Anne-Laure Boulesteix, Difan Deng, Marius Lindauer

Abstract: After a general introduction of hyperparameter optimization, we review important HPO methods such as grid or random search, evolutionary algorithms, Bayesian optimization, Hyperband and racing. We in...