Experiencing Annotation: Emotion, Motivation and Bias in Annotation Tasks

Teodor Stoev, Kristina Yordanova, Emma L. Tonkin

Abstract: In this paper we present different aspects of how emotions can relate to the problem of data annotation. Our main contribution is an extensive literature review where we discuss how emotions could influence different stages of annotation tasks. The results of our literature review show that the nature of the data itself might be crucial to how the emotional state of annotators is affected, but also that the process is affected by other factors such as initial motivation, time pressure, repetitiveness, emotional reaction to particular words/scenes, and demographic background of annotators. In addition to the literature review we formulate several hypotheses connected with the effects of emotions on the overall annotation process and propose an experiment with the help of which these hypotheses could be studied.