Data Driven Learning of Aperiodic Nonlinear Dynamic Systems Using Spike Based Reservoirs-in-Reservoir

Ankita Paul, Nagarajan Kandasamy, Kapil R. Dandekar, Anup Das

Abstract: Mimicking an aperiodic nonlinear dynamic system is challenging as it is difficult to represent it using closed-form equations. A feedback-driven spike-based recurrent spiking neural network is a powerful computational model that can mimic such dynamical systems. We propose reservoirs-in-reservoir (R-i-R), a novel architecture to mimic the frequent pattern changes in space and time of an aperiodic nonlinear dynamic system. Here, a large reservoir is built by connecting multiple small reservoirs to a common output. These small reservoirs are individually specialized to mimic a portion of the input dynamic. The internal recurrent connections of each reservoir and its readout are trained using a recursive least squares (RLS)-based full first-order and reduced control error (full-FORCE) algorithm. To make the entire R-i-R architecture adaptable to the change in periodicity of an input, we implement a new cost function that incorporates a unique forgetting factor to control the fading and wind-up of the covariance matrix of each reservoir during training. We evaluate R-i-R using seven aperiodic nonlinear dynamic systems. We show that R-i-R with rate encoding reduces the error rate by an average 59% with 1.8X reduction in network size compared to state-of-the-art. To improve energy efficiency, we implement a time-to-first-spike encoding and show an average reduction of 39. 5% in the number of spikes.