Gradient-based Hyperparameter Optimization without Validation Data for Learning fom Limited LabelsDownload PDF


Sep 29, 2021 (edited Oct 05, 2021)ICLR 2022 Conference Blind SubmissionReaders: Everyone
  • Keywords: hyperparameter optimization, learning from limited labels, bayesian model selection
  • Abstract: Optimizing hyperparameters of machine learning algorithms especially for limited labeled data is important but difficult, because then obtaining enough validation data is practically impossible. Bayesian model selection enables hyperparameter optimization \emph{without validation data}, but it requires Hessian log determinants, which is computationally demanding for deep neural networks. We study methods to efficiently approximate Hessian log determinants and empirically demonstrate that approximated Bayesian model selection can effectively tune hyperparameters of algorithms of deep semi-supervised learning and learning from noisy labels.
  • One-sentence Summary: We compare efficient methods to approximate Bayesian model selection and show their applications in limited-labeled learning
0 Replies