Enhanced Blind Image Restoration with Channel Attention Transformers and Multi-Scale Attention Prompt-based Learning

Jianhua hu, K. L. Eddie Law

Abstract:

Deep learning models today are indispensable tools for image compression and restoration. However, despite recent progress, many existing models often lack generalization upon facing with different types and coding strength designs of image restoration, thus limiting their practical application. In this paper, a novel approach called {\em dual-Channel Transformers and Multi-scale attention Prompt learning (CTMP)} is introduced to bridge the gap on blind image restoration. The prompt-based learning approach is employed in the model to address two key image restoration tasks: 1) compressed image artifact removal, and 2) image denoising. By utilizing adaptive prompts to accommodate varying quantization parameter (QP) values and noise conditions, and enhancing adaptability through the integration of multi-scale attention mechanisms, an advanced Transformer architecture in our model can tackle diverse image degradations in blind image restoration. That is, our Transformer module is improved through merging and harnessing the strengths of both channel attention and self-attention. The design is adept at extracting both high-frequency details and low-frequency structures, thereby significantly enhancing overall restoration performance. Using the Kodak dataset in experiments, our model outperforms conventional deep learning techniques with a 2.44\% reduction of BD-rate in blind mode. It shows a 29.21\% improvement over traditional JPEG compression and a 0.14 dB improvement in blind denoising. The experiments demonstrate that our approach is capable of training a single model effectively for both compressed image artifact removal and image denoising.
The code is publicly available on GitHub at https://github.com/gdit-ai/CTMP.

Submission Number: 101