Towards Context-Based Retrieval in Associative Memories

Moulik Choraria, Argyrios Gerogiannis, Vidhata Jayaraman, Ankur Mani, Lav R. Varshney

Keywords: Context-based Retrieval, Associative Memories

TL;DR: We propose a two-stage energy-based associative memory where context reshapes the retrieval landscape, provably increasing separation and inducing sparsity for improved recall.

Abstract:

Hopfield networks and their generalizations have established deep connections between biological associative memories, statistical physics, and transformers. Yet most models treat retrieval as a fixed query-to-memory mapping, ignoring the role of external context in recall. In this work, we propose a two-stage associative memory architecture within the modular energy framework of hierarchical associative memories. Herein, a context-gate sub-circuit reshapes the retrieval energy landscape before and during recall.
We demonstrate how this structure can increase inter-memory separation and exponentially improve retrieval, while actively inducing sparsity over the space of memories available for recall. Our framework offers a promising step towards a principled account of how external context can reshape retrieval dynamics.

Submission Number: 46