Hyper-Parameter Optimization for Improving the Performance of Grammatical Evolution

Hao Wang, Yitan Lou, Thomas Bäck

Abstract: State-of-the-art Grammatical Evolution systems such as PonyGE2 have a number of hyper-parameters that control the behavior of the internal evolutionary algorithm for evolving the representations of programs. In this paper, a variant of the efficient global optimization (EGO) algorithm is applied for optimizing these hyper-parameters of the PonyGE2-system. This approach is tested on four test problems used in the Grammatical Evolution community: StringMatch, symbolic regression (the `Vladislavleva-4' problem), bank note classification and the so-called Pymax task. The experimental results show that the average performance of the GE system is improved significantly (between 25% and 168%) on all of the test problems. In addition, the resulting overall best hyper-parameter settings are substantially different from the defaults used in PonyGE2.