Sequential Pattern Retrieval: New Representations Inspired by Non-equilibrium Physics and Associative Memory Models

Matthew Du, Agnish Kumar Behera, Madan Rao, Suriyanarayanan Vaikuntanathan

Track: long paper (up to 5 pages)

Keywords: Hopfield model, energy-based model, long short-term-memory, sequence generation, Modern Hopfield networks, recurrent neural networks

TL;DR: We report strategies inspired by noneqilibrium physics and modern associative memory models to improve retrieval of temporal sequences.

Abstract:

Generating a temporal sequence of outputs from a single output has broad relevance, including in neuroscience and machine learning.
Inspired by ideas in non-equilibrium physics and modern associative memory models, we demonstrate new representations of sequence recall. Our findings provide potential strategies to improve the learning of temporal data in state-of-the-art neural networks.

Submission Number: 26