Using Motion Compensation and Matrix Completion Algorithm to Remove Rain Streaks and Snow for Video Sequences

Yutong Zhou, Nobutaka Shimada

Abstract: The current outdoor surveillance equipment and cameras are vulnerable to be influenced by rain, snow, and other inclement weather, reducing the performance of the surveillance systems. In this paper, we propose a method to detect and remove rain streaks even snow artifacts from video sequences, using motion compensation and low-rank matrix completion method. First, we adopt the optical flow estimation method between consecutive frames to get a warped frame and obtain an initial binary rain map. We further use morphological component analysis method to dilate the tiny rain streaks. Then we employ the online dictionary learning for sparse representation technique and SVM classifier to refine the rain map by getting rid of parts which are not rain streaks. Finally, we reconstruct the video sequence by using low-rank matrix completion techniques. The experimental results demonstrate the proposed algorithm and perform qualitatively as well as quantitatively better in terms of PSNR/SSIM.