Joint Video Denoising and Super-Resolution Network for IoT Cameras

Liming Ge, Wei Bao, Xinyi Sheng, Dong Yuan, Bing Bing Zhou, Zhiyong Wang

Abstract: Internet of Things (IoT) cameras have widely been deployed over the last few years. These cameras are often with limited hardware so that they can only capture noisy videos in low resolution. In this work, we propose the joint video denoising and super-resolution network for IoT cameras, which consists of the noise-robust moving-attention (NRMA) module and the noise-eliminated upsampling (NEU) module. In NRMA, we adopt a coarse-to-fine approach by first extracting the coarse flow and then refining through bi-directional feature propagation among adjacent frames. In NEU, we further utilize inner-frame features for noise-elimination and upsampling. Through this approach, we avoid the negative effects brought by applying denoising and super-resolution in tandem, and enhance the reconstruction of moving objects by the embedded attention layers in NRMA. We conduct our experiments on both synthetic data sets, which utilize existing data with additive white Gaussian noise (AWGN), and a realistic data set captured using a pair of IoT and professional cameras. Our extensive experimental results demonstrate that our proposed method significantly reduces noise and enhances detail in both types of data sets. Notably, our approach outperforms the state-of-the-art benchmark (RealBasicVSR) by an average of 5.24 dB on the existing data sets (with noise level $\sigma{=}20$ ) and by 0.95 dB on the realistic data set in terms of peak signal to noise ratio.