ISP-GAN: inception sub-pixel deconvolution-based lightweight GANs for colorization

Long Zhuo, Shunquan Tan, Bin Li, Jiwu Huang

Abstract: Though there are many encouraging reports, existing image colorization algorithms are still prone to unnatural visual distortions. We observe that unnatural visual distortions are mainly introduced in the deconvolutional modules of existing generative models. Furthermore, the existing algorithms are with heavily structures, which hinders the deployment of algorithms on edge devices. In this paper, we propose ISP-GAN, a novel lightweight generative adversarial network with inception sub-pixel deconvolution aimed at improving the performance of image colorization. In the generator of our proposed ISP-GAN, we propose a novel inception sub-pixel deconvolutional block (ISP), along with a modified residual convolutional block (MRC), to avoid representational bottlenecks and consequently expand perceptual fields. For the ISP-GAN discriminator, we apply deep-learning-based steganalytic networks to improve the training efficiency of the whole framework and consequently enhance the performance of the corresponding generator. Our ISP-GAN is with lightweight structures and experimental results on the benchmark datasets show that ISP-GAN can achieve state-of-the-art performance in the image colorization task.