Offensiveness as an Opinion: Dissecting population-level Label Distributions

Tharindu Cyril Weerasooriya, Sarah Luger, Yu Liang, Christopher M Homan

Keywords: human annotation representations, ethical AI, natural language representations

TL;DR: study calling to action the need for different levels of analysis into population-level label distributions

Abstract: Human annotation is an essential component for building human-in-the-loop machine learning systems (MLs). The diverse human disagreement that arises during annotation is often obscured because of majority voting label aggregation used for training MLs. When the minority opinion is removed in this process it may also extricate the sentiments held by people in minority demographics. This information is essential when MLs are used for offensive or hate speech identification as some content is offensive to only a minority. Collecting human annotations is an expensive task and it is even more challenging when collecting for minority voices. Population-level learning (PLL) utilizes unsupervised learning methods to represent populations of annotators using existing annotations. We test the viability and transparency of PLL with a large dataset of toxic content. We explore the clusters qualitatively by studying the language of the data items assigned to different clusters. In addition, we quantitatively analyze the nature of human disagreement via the data points assigned to the clusters.