Stereo Image Compression Using Recurrent Neural Network With A Convolutional Neural Network-Based Occlusion Detection

Muhammad Shahzeb Khan Gul, Hamid Suleman, Michel Bätz, Joachim Keinert

Abstract: In this work, we propose an end-to-end trainable recurrent neural network for stereo image compression. The recurrent neural network allows variable compression rates without retraining the network due to the iterative nature of the recurrent units. The proposed method leverages the fact that stereo images have overlapping fields of view, i.e., mutual information, to reduce the overall bit rate. Each image in the stereo pair has its separate encoder and decoder network. We propose to share the mutual information between the stereo pair networks by warping the hidden states of one of the stereo image network’s recurrent layers to the other stereo image network’s recurrent layers. Moreover, we also improve the quality of the shared mutual information by eliminating the wrong information by estimating occlusion maps using a convolutional neural network. The proposed method results show significant bit rate savings compared to the single image compression baseline model and traditional codecs.