Hyperparameter Optimization of LSTM Network Models through Genetic Algorithm

Nikolaos Gorgolis, Ioannis Hatzilygeroudis, Zoltán Istenes, Lazlo-Grad Gyenne

Abstract: Next word prediction is an important problem in the domain of NLP, hence in modern artificial intelligence. It draws both scientific and industrial interest, as it consists the core of many processes, like autocorrection, text generation, review prediction etc. Currently, the most efficient and common approach used is classification, using artificial neural networks (ANNs). One of the main drawbacks of ANNs is fine - tuning their hyperparameters, a procedure which is essential to the performance of the model. On the other hand, the approaches usually used for fine - tuning are either computationally unaffordable (e.g. grid search) or of uncertain efficiency (e.g. trial & error). As a response to the above, through the current paper is presented a simple genetic algorithm approach, which is used for the hyperparameter tuning of a common language model and it achieves tuning efficiency without following an exhaustive search.