Noise Robust Video Super-Resolution Without Training on Noisy Data

Fei Zhou, Zitao Lu, Hongming Luo, Cuixin Yang, Bozhi Liu

Abstract: Previous CNN-based video super-resolution (VSR) approaches can not be directly applied to noisy images, otherwise the noise will be enhanced after super-resolution (SR) reconstruction models. Some methods are robust to noise but all of them need to be trained on specific noisy training datasets. In this paper, we propose a noise-robust VSR network which only needs to be trained on the clean images. That is, in our deep network for VSR, the model can appropriately super-resolve noisy images without any training on noisy data. We put forward a non-local spatio-temporal module, which not only achieves motion estimation and compensation, but also improves the robustness of our VSR model to noise. A inter-frame fusion module is further presented to fuse the complementary information from different frames. The experiments conducted on both additive noise and multiplicative noise demonstrate that the proposed method can generate visually and quantitatively high-quality results, superior to state-of-the-art methods.