Accuracy-Time Efficient Hyperparameter Optimization Using Actor-Critic-based Reinforcement Learning and Early Stopping in OpenAI Gym Environment

Albert Budi Christian, Chih-Yu Lin, Yu-Chee Tseng, Lan-Da Van, Wan-Hsun Hu, Chia-Hsuan Yu

Abstract: In this paper, we present accuracy-time efficient hyperparameter optimization (HPO) using advantage actor-critic (A2C)-based reinforcement learning (RL) and early stopping in OpenAI Gym environment. The A2C RL can improve the hyperparameter selection such that the resulting accuracy of machine learning (ML) algorithms including XGBoost, support vector classifier (SVC), random forest shows comparable. According to the specified accuracy of the ML algorithms, the early stopping scheme can save the computation cost. Ten standard datasets are used to valid the accuracy-time efficient HPO. Experimental results show that the presented accuracy-efficient HPO architecture can improve 0.77% accuracy on average compared with default hyperparameter for random forest. The early stopping can save 64% computation cost on average compared to without early stopping for random forest.