$1\times1$ Convolution is All You Need for Image Super-Resolution

Gang Wu, Junjun Jiang, Xianming Liu

Keywords: Single Image Super-Resolution, Lightweight Single Image Super-Resolution

TL;DR: We exploit and demonstrate the effectiveness of the $1\times1$ convolution with spatial-shift in SR task.

Abstract:

In resource-constrained environments, such as mobile devices, lightweight and efficient architectures are crucial for the deployment of single image super-resolution (SISR) deep models. Due to the advantage of achieving a good trade-off between model capacity and efficiency, $3\times3$ convolutions are widely utilized in current convolutional neural networks (CNN). Compared to the normal $3\times3$ convolution, $1\times1$ convolution involves less computation burden but lacks the ability to represent and aggregate spatial information. Accordingly, a common sense in the literature is that $1\times1$ solely cannot constitute a powerful SR network. In this paper, we revisit $1\times1$ in the lightweight scenario and demonstrate that the fully $1\times1$ convolutional network with strong learning ability can be achieved for SISR, thanks to the manual spatial-shift operation. We investigate the feature aggregation scheme in normal $3\times3$ convolution and analogously extend the $1\times1$ convolution with a parameter-free spatial-shift operation, simplified as the shift-conv layer.
In the proposed SISR method, we replace all normal $3\times3$ convolutions with shift-conv layers and present the $\mathbf{S}$hift-$\mathbf{C}$onv-based $\mathbf{N}$etwork (SCNet). Extensive experiments demonstrate that SCNets with all $1\times1$ convolutions obtain even better results than SR models with normal $3\times3$ convolutions that have a larger model size.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Applications (eg, speech processing, computer vision, NLP)