TC-net: transformer combined with cnn for image denoising

Tao Xue, Pengsen Ma

Abstract: In this paper, we propose an effective and novel network architecture based on the transformer TC-Net. The architecture is composed of several transformer blocks and convolutions for image denoising. The following four core designs of TC-Net ensure that it is suitable for image denoising: (1) An extra skip-connection for feature fusion ensures more effective transmission and utilization of low-level and high-level features. (2) Window multihead self-attention greatly reduces the amount of calculation and captures the dependence of capturing long distances. (3) A convolution-based forward network further improves the ability to capture local information. (4) Ingeniously adding a deep residual shrinkage network into a transformer block improves the networks ability to deal with noise and its robustness in complex scenes. Not just for denoising tasks. To deal with similar low-level visual tasks, only the dataset needs to be changed. The model architecture remains the same and is trained separately. Then, the pretraining models of other tasks can be obtained. A large number of experiments proved the ability of TC-Net in image restoration and demonstrated its efficiency and effectiveness in various scenes.