Learning image compressed sensing with sub-pixel convolutional generative adversarial network

Yubao Sun, Jiwei Chen, Qingshan Liu, Guangcan Liu

Abstract: Highlights • The proposed SCGAN learns an explicit mapping from the compressed measurement to the reconstruction in an adversarial manner, thereby the reconstruction to any testing sample can be obtained by simply feeding the measurement vector into the well-trained generator, improving significantly the reconstruction quality and speeding up the CS reconstruction procedure greatly. • Our generator network uses multiple sub-pixel convolutions to progressively upscale the dimension of feature maps until reaching the dimension of the original image. The sub-pixel convolutions can extract more feature maps for resolution upscaling and thus promote the reconstruction quality. • We design a compound loss function, consisting of CS task-oriented reconstruction loss, measurement loss term and Wasserstein adversarial loss term, to encourage the output of generator to have a similar statistical distribution as the real images. Abstract Compressed sensing (CS) is a new technology to reconstruct image from randomized measurements, but the reconstruction procedure involves a time-consuming iterative optimization. In addition, the reconstruction quality becomes poor in low sampling rate. In order to alleviate these issues of the conventional CS image reconstruction, we propose a novel sub-pixel convolutional generative adversarial network (GAN) to learn compressed sensing reconstruction of images. The generator constructs the sub-pixel convolutional network to learn the explicit mapping from the low-dimensional measurement vector to the high-dimensional reconstruction, in which a compound loss, including reconstruction loss, measurement loss and adversarial loss, is designed to guide the network learning. By means of the adversarial training with discriminator, the generator can learn the inherent image distribution and improve the reconstruction quality. Moreover, the test image can be fast reconstructed by simply passing the low-dimensional measurement vector through the generator network. The proposed algorithm is tested on MNIST, F-MNIST and CelebA datasets, and the experimental results show that it is superior to some state-of-the-art deep learning based and iterative optimization based algorithms, in terms of both time complexity and reconstruction quality.