Open Peer Review. Open Publishing. Open Access. Open Discussion. Open Directory. Open Recommendations. Open API. Open Source.
Online Hyper-Parameter Optimization
Damien Vincent, Sylvain Gelly, Nicolas Le Roux, Olivier Bousquet
Feb 15, 2018 (modified: Feb 15, 2018)ICLR 2018 Conference Blind Submissionreaders: everyoneShow Bibtex
Abstract:We propose an efficient online hyperparameter optimization method which uses a joint dynamical system to evaluate the gradient with respect to the hyperparameters. While similar methods are usually limited to hyperparameters with a smooth impact on the model, we show how to apply it to the probability of dropout in neural networks. Finally, we show its effectiveness on two distinct tasks.
TL;DR:An algorithm for optimizing regularization hyper-parameters during training
Enter your feedback below and we'll get back to you as soon as possible.