Neural Stored-program Memory

Sep 25, 2019 Blind Submission readers: everyone Show Bibtex
  • Abstract: Neural networks powered with external memory simulate computer behaviors. These models, which use the memory to store data for a neural controller, can learn algorithms and other complex tasks. In this paper, we introduce a new memory to store weights for the controller, analogous to the stored-program memory in modern computer architectures. The proposed model, dubbed Neural Stored-program Memory, augments current memory-augmented neural networks, creating differentiable machines that can switch programs through time, adapt to variable contexts and thus fully resemble the Universal Turing Machine. A wide range of experiments demonstrate that the resulting machines not only excel in classical algorithmic problems, but also have potential for compositional, continual, few-shot learning and question-answering tasks.
  • Keywords: Memory Augmented Neural Networks, Universal Turing Machine, fast-weight
  • TL;DR: A neural simulation of Universal Turing Machine
  • Code: https://github.com/thaihungle/NSM
  • Original Pdf:  pdf
0 Replies

Loading