Binarized Low-Light Raw Video Enhancement

Gengchen Zhang, Yulun Zhang, Xin Yuan, Ying Fu

Abstract: Recently, deep neural networks have achieved excellent performance on low-light raw video enhancement. How-ever, they often come with high computational complexity and large memory costs, which hinder their applications on resource-limited devices. In this paper, we explore the feasibility of applying the extremely compact binary neural network (BNN) to low-light raw video enhancement. Nev-ertheless, there are two main issues with binarizing video enhancement models. One is how to fuse the temporal in-formation to improve low-light denoising without complex modules. The other is how to narrow the performance gap between binary convolutions with the full precision ones. To address the first issue, we introduce a spatial-temporal shift operation, which is easy-to-binarize and effective. The temporal shift efficiently aggregates the features of neigh-bor frames and the spatial shift handles the misalignment caused by the large motion in videos. For the second issue, we present a distribution-aware binary convolution, which captures the distribution characteristics of real-valued in-put and incorporates them into plain binary convolutions to alleviate the degradation in performance. Extensive quantitative and qualitative experiments have shown our high-efficiency binarized low-light raw video enhancement method can attain a promising performance. The code is available at https://github.com/ying-fuIBRVE.