Multi-Source Unsupervised Hyperparameter Optimization

Masahiro Nomura, Yuta Saito

Keywords: Hyperparameter Optimization

Abstract: How can we conduct efficient hyperparameter optimization for a completely new task? In this work, we consider a novel setting, where we search for the optimal hyperparameters for a target task of interest using only unlabeled target task and ‘somewhat relevant’ source task datasets. In this setting, it is essential to estimate the ground-truth target task objective using only the available information. We propose estimators to unbiasedly approximate the ground-truth with a desirable variance property. Building on these estimators, we provide a general and tractable hyperparameter optimization procedure for our setting. The experimental evaluations demonstrate that the proposed framework broadens the applications of automated hyperparameter optimization.

One-sentence Summary: We enable efficient hyperparameter optimization for an unlabeled task by leveraging information on related tasks.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Community Implementations: [ 5 code implementations](https://www.catalyzex.com/paper/arxiv:2006.10600/code)

Reviewed Version (pdf): https://openreview.net/references/pdf?id=xbMXJ3iDQW