Real-time Night Surveillance Video Retrieval through Calibrated Denoising and Super-resolution

Liming Ge, Wei Bao, Xinyi Sheng, Dong Yuan, Bing Bing Zhou

Abstract: Real-time video surveillance cameras have been widely deployed over the last few years. In case of incidents such as natural disasters, it provides vital guidance in real time to aid the rescue operations. However, the quality of the captured video is far from satisfactory due to the limited camera hardware and low network bandwidth. Noise is often observed especially at night and the resolution is low. To this end, we are motivated to retrieve the nighttime surveillance video through calibrated denoising and super-resolution. We only use the preceding and current frames, but not the future frames. Thus, we avoid the additional delay of waiting for future frames, which is not suitable for real-time video applications. We design the pipeline semantically beneficial for both denoising and super-resolution, and achieve high rendering quality, especially for real-world noise. Moreover, we propose a novel calibration method for collecting paired noisy and clean observations in the real world, which provides more effective training data. We conduct experiments using the real-world dataset collected under low-light conditions, and benchmark with state-of-the-art video denoising and super-resolution methods. Results show that our method achieves significant performance gain while introducing small delay compared with the benchmarks, suitable for real-time videos.