Multi-distorted Image Restoration with Tensor 1 × 1 Convolutional Layer

Zihao Huang, Chao Li, Feng Duan, Qibin Zhao

Abstract: Image restoration with corruptions from combined multiple types of distortion is a challenging and practical problem. Recent studies show that a promising technique is to perform parallel “operations” to handle different types of distortion, which can be modeled by a deep neural network framework. However, the reconstruction may be dominated by a small number of operations due to the heterogeneous features generated by different operations. To handle this issue, we introduce a tensor 1×1 convolutional layer by leveraging high-order tensor fusion, which can not only harmonize the heterogeneous features but also take high order statistical information into account. To efficiently learn the large-scale kernel tensor resulted from the tensor product, we employ tensor network to represent kernels, which is able to convert the exponential growth of the dimension to linear growth. Armed with this new layer, we propose high-order operation-wise attention network for the task of multi-distorted image restoration. The experimental results demonstrated that the proposed method outperforms the method with vanilla 1 × 1 convolutional layer in several typical tasks and is promising for more difficult tasks. Code is available at https://github.com/ZihaoH/High-order-OWAN.