Progressive Local and Non-Local Interactive Networks with Deeply Discriminative Training for Image Deraining

Cong Wang, Liyan Wang, Jie Mu, Chengjin Yu, Wei Wang

Abstract: In this paper, we develop a progressive local and non-local interactive network with multi-scale cross-content deeply discriminative learning to solve image deraining. The proposed model contains two key techniques: 1) Progressive Local and Non-Local Interactive Network (PLNLIN) and 2) Multi-Scale Cross-Content Deeply Discriminative Learning (MCDDL). The PLNLIN is a U-shaped encoder-decoder network, where the proposed new Progressive Local and Non-Local Interactive Module (PLNLIM) is the basic unit in the encoder-decoder framework. The PLNLIM fully explores local and non-local learning in convolution and Transformer operation respectively and the local and non-local content are further interactively learned in a progressive manner. The proposed MCDDL not only discriminates the output of the generator but also receives the deep content from the generator to distinguish real and fake features at each side layer of the discriminator in a multi-scale manner. We show that the proposed MCDDL has fast and stable convergence properties that lack in existing discriminative learning manners. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art methods on five public synthetic datasets and one real-world data. The source codes will be made available at \url{https://github.com/supersupercong/PLNLIN-MCDDL}.

Primary Subject Area: [Experience] Multimedia Applications

Relevance To Conference: rainy images captured in outdoor surveillance equipment may significantly degrade the performance of some existing computer vision systems and may also result in a pool of visual experience for some multimedia applications. Hence, improving the visual quality helps achieve better accuracy for these multimedia applications.

Submission Number: 2270