A Gated Recurrent Unit based Echo State Network

Xinjie Wang, Yaochu Jin, Kuangrong Hao

Abstract: Echo State Network (ESN) is a fast and efficient recurrent neural network with a sparsely connected reservoir and a simple linear output layer, which has been widely used for real-world prediction problems. However, the capability of the ESN of handling complex nonlinear problems is limited by the relatively simple neuronal dynamics in the reservoir. Although the gated recurrent unit (GRU) model with multiple nonlinear operators has achieved an excellent performance, gradient-based training algorithms usually require intensive computational resources. In this paper, we present a novel ESN model based on GRUs to tackle complex real-world tasks while reducing the computational costs, taking advantage of the characteristics of both the ESN and the GRU models. In the proposed model, the reservoir unit is replaced by the sparsely connected GRU neurons. Experimental results on three regression problems demonstrate that the proposed method performs better than the original ESN and GRU models.