Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameters

Jelena Luketina, Tapani Raiko, Mathias Berglund, Klaus Greff

Abstract: Hyperparameter selection generally relies on running multiple full training trials, with selection based on validation set performance. We propose a gradient-based approach for locally adjusting hy...