Frequency-Aware Physics-Inspired Degradation Model for Real-World Image Super-Resolution

Zhenxing Dong

Abstract:

Current learning-based single image super-resolution (SISR) algorithms underperform on real data due to the deviation in the assumed degradation process from that in the real-world scenario. Conventional degradation processes consider applying blur, noise, and downsampling (typically bicubic downsampling) on high-resolution (HR) images to synthesize lowresolution (LR) counterparts. However, few works on degradation modelling have taken the physical aspects of the optical imaging system into consideration. In this paper, we analyze the imaging system optically and exploit the characteristics of the real-world LR-HR pairs in the spatial frequency domain. We formulate a real-world physics-inspired degradation model by considering both optics and sensor degradation; The physical
degradation of an imaging system is modelled as a low-pass filter, whose cut-off frequency is dictated by the object distance, the focal length of the lens, and the pixel size of the image sensor. In particular, we propose to use a convolutional neural network (CNN) to learn the cutoff frequency of real-world degradation process. The learned network is then applied to synthesize LR images from unpaired HR images. The synthetic HRLR image pairs are later used to train an SISR network. We evaluate the effectiveness and generalization capability of the proposed degradation model on real-world images captured by different imaging systems.