Robust Foreground Segmentation in RGBD Data from Complex Scenes Using Adversarial Networks

Maryam Sultana, Thierry Bouwmans, Jhony H. Giraldo, Soon Ki Jung

Abstract: Foreground segmentation is a fundamental problem in many artificial intelligence and computer vision based applications. However, robust foreground segmentation with high precision is still a challenging problem in complex scenes. Currently, many of the existing algorithms process the input data in RGB space only, where the foreground segmentation performance is most likely degraded by various challenges like shadows, color camouflage, illumination changes, out of range camera sensors and bootstrapping. Cameras capturing RGBD data are highly active visual sensors as they provide depth information along with RGB of the given input images. Therefore, to address the challenging problem we propose a foreground segmentation algorithm based on conditional generative adversarial networks using RGB and depth data. The goal of our proposed model is to perform robust foreground segmentation in the presence of various complex scenes with high accuracy. For this purpose, we trained our GAN based CNN model with RGBD input data conditioned on ground-truth information in an adversarial fashion. During training, our proposed model aims to learn the foreground segmentation on the basis of cross-entropy loss and euclidean distance loss to identify between real vs fake samples. While during testing the model is given RGBD input to the trained generator network that performs robust foreground segmentation. Our proposed method is evaluated using two RGBD benchmark datasets that are SBM-RGBD and MULTIVISION kinect. Various experimental evaluations and comparative analysis of our proposed model with eleven existing methods confirm its superior performance.