The Capacity of the Dense Associative Memory Networks

HAN BAO, Richong Zhang, Yongyi Mao

Abstract:

This paper revisits the dense associative memory (DAM) networks and studies rigorously the capacity of
the DAM networks. We present the capacity theorem of the DAM networks with an attraction radius or a
noise level from the messages and prove that the probe can converge to the targeted message just after
the one-step update. Under this convergence, the capacity of DAM networks is between a lower bound
and an upper bound. Although when the attraction radius is 0:0 away from the messages, i.e. noiseless,
previous literature provides an approximate result. However, a rigorous proof is not given in this study. In
addition, we consider a more general notion of capacity which allows the retrieval of messages from
noisy probes (the attraction radius is not 0:0). We demonstrates that the convergence result can be
acquired just after the one-step update when the probe is a corrupted version with a Gaussian noise from
one message. We further provide simulated experiments to validate theorems herein.