On the subjectivity and bias of web content credibility evaluations

Michal Kakol, Michal Jankowski-Lorek, Katarzyna Abramczuk, Adam Wierzbicki, Michele Catasta

Abstract: In this paper we describe the initial outcomes of the Reconcile1 study concerning Web content credibility evaluations. The study was run with a balanced sample of 1503 respondents who independently evaluated 154 web pages from several thematic categories. Users taking part in the study not only evaluated credibility, but also filled a questionnaire covering additional respondents' traits. Using the gathered information about socio-economic status and psychological features of the users, we studied the influence of subjectivity and bias in the credibility ratings. Subjectivity and bias, in fact, represent a key design issue for Web Credibility systems, to the extent that they could jeopardize the system performance if not taken into account. We found out that evaluations of Web content credibility are slightly subjective. On the other hand, the evaluations exhibit a strong acquiescence bias.