Predictive Coding, Variational Autoencoders, and Biological ConnectionsDownload PDF

Sep 11, 2019 (edited Oct 23, 2019)NeurIPS 2019 Workshop Neuro AI Blind SubmissionReaders: Everyone
  • TL;DR: connections between predictive coding and VAEs + new frontiers
  • Abstract: Predictive coding, within theoretical neuroscience, and variational autoencoders, within machine learning, both involve latent Gaussian models and variational inference. While these areas share a common origin, they have evolved largely independently. We outline connections and contrasts between these areas, using their relationships to identify new parallels between machine learning and neuroscience. We then discuss specific frontiers at this intersection: backpropagation, normalizing flows, and attention, with mutual benefits for both fields.
  • Keywords: predictive coding, variational autoencoders, probabilistic models, variational inference
4 Replies