Reduced-Gate Convolutional LSTM Design Using Predictive Coding for Next-Frame Video Prediction

Nelly Elsayed, Anthony S. Maida, Magdy Bayoumi

Abstract:

Spatiotemporal sequence prediction is an important problem in deep learning. We
study next-frame video prediction using a deep-learning-based predictive coding
framework that uses convolutional, long short-term memory (convLSTM) modules.
We introduce a novel reduced-gate convolutional LSTM architecture. Our
reduced-gate model achieves better next-frame prediction accuracy than the original
convolutional LSTM while using a smaller parameter budget, thereby reducing
training time. We tested our reduced gate modules within a predictive coding architecture
on the moving MNIST and KITTI datasets. We found that our reduced-gate
model has a significant reduction of approximately 40 percent of the total
number of training parameters and training time in comparison with the standard
LSTM model which makes it attractive for hardware implementation especially
on small devices.

Keywords: rgcLSTM, convolutional LSTM, unsupervised learning, predictive coding, video prediction, moving MNIST, KITTI datasets, deep learning

TL;DR: A novel reduced-gate convolutional LSTM design using predictive coding for next-frame video prediction