National differences in image quality assessment: An investigation on three large-scale IQA datasets

Dietmar Saupe, Simon Hviid Del Pin

Abstract: This paper investigates the potential effects of national differences on image and video quality assessment using discrete rating scales. Drawing on cultural psychology, we hypothesize that observers from different countries may exhibit distinct response styles in interpreting and applying the five-level absolute and degradation category rating scales (ACR, DCR). For our study, we adapt state-of-the-art statistical models for three large-scale image quality datasets (KonIQ-10k, KADID-10k, and NIVD). Our models include country-specific components such as variable rating category thresholds and the probability for extreme ratings on these scales. We found statistically significant differences between ratings collected in different countries. Our results have implications for the analysis and design of current, respectively future datasets and contribute to a more comprehensive understanding of image quality in a global context. We also propose to include lapse rates into statistical models for categorical judgements. Lapse rates model unintentional erroneous responses of subjects in a quality assessment study and provide a regularization mechanism for the scale estimation by maximum likelihood estimation.