Dynamics of modern Hopfield networks

Sofiya Bogakovskaya, Vladimir Fanaskov, Ivan Oseledets

Keywords: associative memory, Hopfield network, dynamical systems, adversarial robustness

Abstract: We empirically analyze the dynamical characteristics of modern Hopfield networks (MHNs) in classification tasks. For bounded activation functions, trajectories are nearly linear, converge to class-specific memory vectors, and exhibit mean-shift dynamics. Conversely, rectified activation functions yield unstable dynamics where memories are instead encoded in the direction of the state vector. Using adversarial examples generated via projected gradient ascent, we show that while certain MHN architectures outperform MLPs in robustness, this trend is not universal. Critically, even robust MHNs possess global basin structures significantly more complex than standard MLP decision boundaries.

Submission Number: 21