Dense Associative Memories with Analog Circuits

Marc Gong Bacvanski, Xincheng You, Dmitry Krotov

Keywords: Dense Associative Memory, AI Hardware, Analog Hardware, Accelerators, Circuits, Energy-based models

TL;DR: We show that Dense Associative Memories can be implemented directly as analog circuits, where inference is the physical relaxation of a resistive crossbar network—and these circuits compose into transformer-like energy-based models.

Abstract: Associative memory models perform inference by converging to fixed points of a dynamical system, making them naturally compatible with analog hardware that computes through physical time evolution. We present a mapping from Dense Associative Memory (DenseAM) dynamics to an analog circuit primitive composed of resistive crossbar arrays and continuous-time neuron integrators. Synaptic weights are implemented as conductances, neuron states are capacitor voltages, and circuit physics directly realize the DenseAM update equations, so that inference corresponds to physical relaxation of the circuit. We illustrate the primitive with a DenseAM that solves the XOR task with attractor dynamics, and show how DenseAM blocks can be composed into an Analog Energy Transformer. On an 8-bit parity task, the Analog Energy Transformer achieves perfect validation accuracy under explicit ODE simulation, and its continuous-time trajectories converge to stable fixed points, yielding robustness to readout timing. These results position DenseAM as a natural computational abstraction for analog AI hardware, unifying associative recall and transformer-like architectures within a single dynamical circuit framework.

Submission Number: 25