A BIOLOGICALLY PLAUSIBLE ASSOCIATIVE MEMORY NETWORK

Mohadeseh Shafiei Kafraj, Dmitry Krotov, Brendan A Bicknell, Peter E. Latham

Track: long paper (up to 5 pages)

Keywords: Associative Memory, Biologically Plausible Models

TL;DR: We propose a biologically plausible associative memory model with polynomial capacity.

Abstract: The Hopfield network has been the leading model for associative memory for over four decades, culminating in the recent 2024 Nobel Prize. However, the vanilla version of the Hopfield network has a capacity that scales with the number of connections per neuron. In the mammalian brain, that’s about 1,000, leading to a capacity of about 50 memories in a spiking network—regardless of its size. Therefore, it cannot possibly account for the capacity of human memory. To address this limitation, various modifications to the Hopfield network have been proposed. One promising variant is Dense Associative Memory, which significantly increases capacity and could be implemented in a two-layer architecture consisting of memory and feature neurons. However, from the point of view of biological plausibility, this comes with a downside: during recall of a specific memory, all neurons but one cued neuron (a neuron that is associated with the recalled memory) in the memory layer are silent, whereas in the brain, neurons are rarely silent for extended periods. This is not easy to fix: the memory layer contains a large number of neurons, and allowing non-cued neurons (neurons that are not associated with the recalled memory) to exhibit even low firing rates can introduce an unacceptable level of noise, preventing the perfect recall of the cued memory. To address this challenge, we propose a novel architecture that introduces nonlinear dendrites in the Dense Associative Memory network. This model supports a capacity that is polynomial in the number of memory neurons while enabling non-cued memory neurons to be unsilenced. The proposed architecture adheres to other key biological constraints, including the presence of both excitatory and inhibitory populations that obey Dale's law and maintain non-saturated firing rates and sparsity in the connections. These properties enhance the model's biological plausibility while achieving polynomial capacity, bridging the gap between theoretical and biological constraints on associative memory.

Submission Number: 13