Self-organization of a Dynamical Orthogonal Basis Acquiring Large Memory Capacity in Modular Reservoir Computing

Yuji Kawai, Jihoon Park, Ichiro Tsuda, Minoru Asada

Abstract: The ability of the brain to generate complex spatiotemporal patterns with a specific timing is essential for motor learning and time series prediction. An approach that tries to replicate this ability using the self-sustained neural activity of a randomly connected recurrent neural network (reservoir) meets the difficulty of orbital instability. We propose a novel system that learns an arbitrary time series as the linear sum (readout) of stable trajectories produced by numerous small network modules. Our experimental results show that the trajectories of the module outputs are orthogonal to each other, that is, the reservoir self-organizes an orthogonal basis. Furthermore, the system can learn the timing of extremely long intervals, say tens of seconds for a millisecond computation unit and the complex time series of the Lorenz system.