Auto Tuning of RNN Hyper-parameters using Cuckoo Search Algorithm

Dhruv Srivastava, Yasha Singh, Anita Sahoo

Abstract: Long Short Term Memory based Recurrent Neural Network (LSTM+RNN) with additional capability of learning long term dependencies in sequential data is an effective model for analyzing time series data. The hyper-parameters play a crucial role in obtaining an optimized learning model for dataset in hand. The comprehensive studies conducted on selecting the network hyper-parameters have tried limited number of combinations and mark this as a limitation. Further, once optimized for a dataset in hand, the model may not be effective for another dataset. This requires auto tuning methods for selecting hyper-parameters. In this paper, meta-heuristic algorithm named Cuckoo Search has been utilized to find suitable heuristics for auto tuning the hyper-parameters of a RNN+LSTM network. The average accuracy of the models optimized for the experimental datasets are 96.3%.