Compressed multi-scale feature fusion network for single image super-resolution

Xinxia Fan, Yanhua Yang, Cheng Deng, Jie Xu, Xinbo Gao

Abstract: Highlights • We propose a multi-scale feature fusion network (MSFF) with structured sparsity for single image Super-resolution. • The MSFF module can discover more possible structure and context information of the input image. • Multiple MSFF modules are cascaded to improve the nonlinear representation ability of the network, and more accurate mapping relationships between the LR and HR image scan be learned. Abstract Recently, deep neural networks have made significant breakthroughs in the image super-resolution (SR) field. Most deep learning-based image SR methods learn an end-to-end network to discover the mapping relationship between low-resolution (LR) and high-resolution (HR) images in order to produce visually satisfactory images. However, these methods only extract a single scale feature to learn the mapping relationship, which will miss some critical information that is required for reconstruction. In this paper, we propose a compressed multi-scale feature fusion (MSFF) network for single image SR. Several MSFF modules are used in the network to extract image features at different scales, which enables us to capture more complete structure and context information of the image for better SR quality. Furthermore, to solve the problems of training difficulty and computational expense consumption caused by the use of the multi-scale structure, structure sparse regularization is designed to learn a MSFF network with a sparse structure and obtain a compressed network, which greatly reduces the network parameters and accelerates the speed whilst sustaining the reconstruction quality. Extensive experiments on a variety of images show that the proposed method can achieve more desirable performance in terms of visual quality than several state-of-the-art methods.