Video Noise Removal Using Progressive Decomposition With Conditional Invertibility

Haoran Huang, Yuhui Quan, Zhenghua Lei, Jinlong Hu, Yan Huang

Abstract: Video denoising aims at removing noise from noisy video frames and meanwhile preserving their structures and details. It is a challenging task, as both noise and video structures/details correspond to high-frequency components of a noisy video which are hard to distinguish. This paper proposes a deep video denoiser using a progressive decomposition process with conditional invertibility. Noisy video frames are first decomposed into two latent codes via a forward process of conditional invertible coupling layers, where one latent code carries the maximal information regarding the noise-free reference frame while the other encodes the information regarding noise, misalignment and content difference. The clean video is then reconstructed from the latent codes of noise-free frames using the reverse pass of the coupling layers. To improve the robustness to variant noise levels, the coupling layers are conditioned on noise level. In addition, memory units are introduced to the conditioned coupling layers to better exploit temporal correlation among frames for feature disentanglement. Experiments on two benchmark datasets have demonstrated the effectiveness of our method.