A Deep Generative Model Imitating Predictive Coding in Human Brain

Anonymous

Keywords: Predictive coding, POMDP, fMRI data

TL;DR: We develop a new human brain-like prediction model by unifying PredNet and TD- VAE, combining both predictive coding and flexible interval prediction abilities in one single model.

Abstract: In recent years, the development of deep generative models for prediction has been attracting attention. In our study, we focus on predictive coding, a concept from the neuroscience literature that hypothesizes the brain is constantly making predictions of sensory input, and attempt to develop a human brain-like prediction model. PredNet is a deep learning model based on the concept of predictive coding, however, it can predict the next image with the same prediction interval, but not with any intervals. On the other hand, Temporal Differential Variational Auto-Encoder (TD-VAE) can predict the next images with any prediction intervals, although it is not a model reflecting human brain function. In this work, we develop a new human brain-like prediction model by unifying PredNet and TD- VAE, combining both predictive coding and flexible interval prediction abilities in one single model. Through experiments on the KITTI Vision Benchmark, we confirmed that our proposed model can predict the next images correctly with flexible prediction intervals. We also investigated the correlation between the feature values of representation layers in the model architecture and human brain activity data evoked under natural video stimulation.