Image Compression-Aware Deep Camera ISP Network

Kwang-Hyun Uhm, Kyuyeon Choi, Seung-Won Jung, Sung-Jea Ko

Abstract: Several recent studies have attempted to fully replace the conventional camera image signal processing (ISP) pipeline with convolutional neural networks (CNNs). However, the previous CNN-based ISPs, simply referred to as ISP-Nets, have not explicitly considered that images have to be lossy-compressed in most cases, especially by the off-the-shelf JPEG. To address this issue, in this paper, we propose a novel compression-aware deep camera ISP learning framework. At first, we introduce a new use case of compression artifacts simulation network (CAS-Net), which operates in the opposite way of commonly used compression artifacts reduction networks. Then, the CAS-Net is connected with an ISP-Net such that the ISP network can be trained with consideration of image compression. Throughout experimental studies, we show that our compression-aware camera ISP network can produce images with a better tradeoff between bit-rate and image quality compared to its compression-agnostic version when the performance is evaluated after JPEG compression.