Zero-shot Diffusion Models for Demosaicing in Bayer and Non-Bayer Image Sensors

Haechang Lee, Dong Ju Mun, Joohoon Lee, Hyunwoo Lee, Se Young Chun

Abstract: As the demand for high-quality, high-resolution images on mobile phones increases, modern devices now incorporate non-Bayer image sensors that maintain some characteristics of the traditional Bayer pattern while introducing new variations. This evolution has increased the complexity of restoring sensor raw images to full RGB using image signal processors (ISPs), revealing the limitations of conventional model-based ISP approaches. To address these challenges, several deep learning models have been proposed for raw image restoration. However, no existing research has utilized a pre-trained diffusion model in a zero-shot manner for this task, particularly across both traditional Bayer and newer non-Bayer sensors. We introduce a novel zero-shot diffusion model framework incorporating our degradation and pseudo-inverse degradation functions for the demosaicing task. Our approach is effectively and universally applicable to a wide range of sensor patterns, showing superior performance, particularly with the latest and future sensors where traditional methods struggle.