Blind image quality assessment for in-the-wild images by integrating distorted patch selection and multi-scale-and-granularity fusion

Jili Xia, Lihuo He, Xinbo Gao, Bo Hu

Abstract: Images taken in natural environments often exhibit complicated distortions, posing significant challenges for assessing their quality. Although current methods prioritize the perception of image contents and distortions, few explicitly investigate local distortions, a crucial factor affecting human visual perception. To mitigate this, this paper proposes a novel blind image quality assessment (IQA) method for in-the-wild images, termed DPSF, which integrates Distorted Patch Selection and multi-scale and multi-granularity feature Fusion. Specifically, it is first explained that the distributions of the mean subtracted contrast normalized coefficients of distorted patches differ from those of undistorted patches. Building upon this, an effective strategy for distorted patch selection is devised. Subsequently, a hybrid Transformer-convolutional neural network (CNN) module is proposed to exploit the benefits of both Transformer and CNN for distortion perception, in which the long-range dependencies of the selected patches are considered. Finally, an effective fusion module is employed for image quality evaluation, amalgamating finer and richer semantic and distortion features from multiple scales and granularities. Experimental results on five authentic IQA databases demonstrate that the proposed method yields more precise quality predictions compared with the state-of-the-art methods.