Model-Based Meta-Reinforcement Learning for Hyperparameter Optimization

Jeroen Albrechts, Hugo Max MARTIN, Maryam Tavakol

Keywords: Hyperparameter optimization, model-based reinforcement learning, meta-learning, state representation

Abstract:

Hyperparameter Optimization (HPO) plays a significant role in enhancing the performance of machine learning models. However, as the size and complexity of (deep) neural architectures continue to increase, conducting HPO has become very expensive in terms of time and computational resources. Existing methods that automate this process still demand numerous evaluations to find the optimal hyperparameter configurations. In this paper, we present a novel approach based on model-based reinforcement learning to effectively improve sample efficiency while minimizing resource consumption. We formulate the HPO task as a Markov decision process and develop a predictive dynamics model for efficient policy optimization. Additionally, we employ the Deep Sets framework to encode the state space, which is then leveraged in meta-learning for transfer of knowledge across multiple datasets, enabling the model to quickly adapt to new datasets. Empirical studies demonstrate that our approach outperforms alternative techniques on publicly available datasets in terms of sample efficiency and accuracy.

Submission Number: 64