Learning Rich Information for Quad Bayer Remosaicing and Denoising

Jun Jia, Hanchi Sun, Xiaohong Liu, Longan Xiao, Qihang Xu, Guangtao Zhai

Abstract: In this paper, we propose a DNNs-based solution to jointly remosaic and denoise the camera raw data in Quad Bayer pattern. The traditional remosaic problem can be viewed as an interpolation process that converts the Quad Bayer pattern to a normal CFA pattern, such as the RGGB one. However, this process becomes more challenging when the input Quad Bayer data is noisy. In addition, the limited amount of data available for this task is not sufficient to train neural networks. To address these issues, we view the remosaic problem as a bayer reconstruction problem and use an image restoration model to remove noises while remosaicing the Quad Bayer data implicitly. To make full use of the color information, we propose a two-stage training strategy. The first stage uses the ground-truth RGGB Bayer map to supervise the reconstruction process, and the second stage leverages the provided Image Signal Processor (ISP) to generate the RGB images from our reconstructed bayers. With the use of color information in the second stage, the quality of reconstructed bayers is further improved. Moreover, we propose a data pre-processing method including data augmentation and bayer rearrangement. The experimental results show it can significantly benefit the network training. Our solution achieves the best KLD score with one order of magnitude lead, and overall ranks the second in Quad Joint Remosaic and Denoise @ MIPI-challenge.