LSRN-AED: lightweight super-resolution network based on asymmetric encoder-decoder

Shuying Huang, Wei Li, Yong Yang, Weiguo Wan, Houzeng Lai

Abstract: Due to limited memory and computing resources, the application of deep neural networks on embedded and mobile devices is still a great challenge. To tackle this problem, this paper proposes a lightweight super-resolution network based on asymmetric encoder–decoder (LSRN-AED), which achieves better performance while reducing model parameters and computation. On the basis of rethinking the roles of encoder and decoder, an asymmetric encoder–decoder (AED) composed of complex encoders and simple decoders is designed to achieve feature extraction and reconstruction. Here, the decoder only adopts one inverted residual block, which can reduce the computational cost of the model and the redundancy of mapping features. For the encoder, inspired by the Transformer structure, an epiphany encoder is designed to realize the feature extraction and representation. In the encoder, a multi-way epiphany attention module (MEAM) is constructed, in which inverted residual blocks are used to replace traditional residual blocks to extract features and reduce model complexity. To realize the selection and enhancement of spatial features, an epiphany attention block (EAB) is designed by exploiting depth-wise convolutions which can learn the significant spatial information of the feature maps. Experimental results demonstrate that the proposed LSRN-AED can achieve better performance at lower parameter cost and outperform some existing state-of-the-art lightweight models. For example, compared to the advanced SMSR method, the proposed LSRN-AED has better evaluation metrics while reducing the number of parameters by 45%, 44%, and 44%, and FLOPs by 44%, 42%, and 41% on the × 2/3/4 SR tasks, respectively. The code will be published on GitHub after our paper is accepted for publication.