Compacter: A Lightweight Transformer for Image Restoration

Zhijian Wu, Jun Li, Yang Hu, Dingjiang Huang

Abstract:

Although deep learning-based methods have made significant advances in the field of image restoration (IR), they often suffer from excessive model parameters. To tackle this problem, this work proposes a compact Transformer (Compacter) for lightweight image restoration by making several key designs. We employ the concepts of projection sharing, adaptive interaction, and heterogeneous aggregation to develop a novel Compact Adaptive Self-Attention (CASA). Specifically, CASA utilizes shared projection to generate Query, Key, and Value to simultaneously model spatial and channel-wise self-attention. The adaptive interaction process is then used to propagate and integrate global information from two different dimensions, thus enabling omnidirectional relational interaction. Finally, a depth-wise convolution is incorporated on Value to complement heterogeneous local information, enabling global-local coupling. Moreover, we propose a Dual Selective Gated Module (DSGM) to dynamically encapsulate the globality into each pixel for context-adaptive aggregation. Extensive experiments demonstrate that our Compacter achieves state-of-the-art performance for a variety of lightweight IR tasks with approximately 400K parameters.

Primary Subject Area: [Experience] Interactions and Quality of Experience

Secondary Subject Area: [Experience] Multimedia Applications

Relevance To Conference: This paper presents a lightweight neural network for image restoration task. This is useful for improving the display of multimedia information on high-resolution devices such as TVs and smartphones to enhance the user's usage and interaction experience.

Submission Number: 459