Randomly Coupled Oscillators for Time Series Processing

Andrea Ceni, Andrea Cossu, Jingyue Liu, Maximilian Stölzle, Cosimo Della Santina, Claudio Gallicchio, Davide Bacciu

Keywords: recurrent neural networks, dynamical systems, reservoir computing

TL;DR: We introduce Randomly Coupled Oscillators, a physically-inspired recurrent model based on reservoir computing principles. We derive conditions for its stability and evaluate its effectiveness on time series benchmarks.

Abstract: We investigate a physically-inspired recurrent neural network derived from a continuous-time ODE modelling a network of coupled oscillators. Enthralled by the Reservoir Computing paradigm, we introduce the Randomly Coupled Oscillators (RCO) model, which leverages an untrained recurrent component with a smart random initialization. We analyse the architectural bias of RCO and its neural dynamics. We derive sufficient conditions for the model to have a unique asymptotically uniformly stable input-driven solution. We also derive necessary conditions for stability, that permit to push the system of oscillators slightly beyond the edge of stability. We empirically assess the effectiveness of RCO in terms of its stability and its long-term memory properties. We compare its performance against both fully-trained and randomized recurrent models in a number of time series processing tasks. We find that RCO provides an excellent trade-off between robust long-term memory properties and ability to predict the behavior of non-linear, chaotic systems.

Submission Number: 43