The Capacity of Modern Hopfield Networks under the Data Manifold Hypothesis

Beatrice Achilli, Luca Ambrogioni, Carlo Lucibello, Marc Mezard, Enrico Ventura

Track: long paper (up to 5 pages)

Keywords: Modern Hopfield Networks, manifold, generalization

TL;DR: We study the capacity of the exponential Hopfield model for binary patterns and patterns generated from a hidden manifold model.

Abstract: We generalize the computation of the capacity of exponential Hopfield model from Lucibello and Mézard (2024) to more generic pattern ensembles, including binary patterns and patterns generated from a hidden manifold model.

Submission Number: 19