Tracing Footprints: Neural Networks Meet Non-integer Order Differential Equations For Modelling Systems with Memory

C. Coelho, M.Fernanda P. Costa, Luís L. Ferrás

Keywords: Differential Equations; Neural Networks; Memory; Long-term Dependencies; Time-series; Neural ODEs

TL;DR: Neural Fractional Differential Equation, a neural network that fits a FDE to data inherently exhibiting memory.

Abstract: Neural Ordinary Differential Equations (Neural ODEs) have gained popularity for modelling real-world systems, thanks to their ability to fit ODEs to data. However, numerous systems in science and engineering often exhibit intricate memory behaviours, being classical ODEs inadequate for such tasks due to their inability to handle strong and complex memory effects. In this work, we introduce the Neural Fractional Differential Equation (Neural FDE), a Neural Network (NN) architecture to fit a FDE to data. With this we leverage the capabilities of FDEs allowing the architecture to take into account all past states and their influence on a system's current and future behaviours. Neural FDE inherently exhibits memory, providing a more accurate representation of complex phenomena in systems with long-term dependencies. Numerical experiments show Neural FDE generalises better and has faster convergence than Neural ODEs.

Submission Number: 49