A Framework to Assess (Dis)agreement Among Diverse Rater Groups

Anonymous

TL;DR: We propose a comprehensive disagreement analysis framework to measure systematic diversity in perspectives among different rater subgroups.

Abstract: Human annotation plays a core role in machine learning — annotations for supervised models, safety for generative models, and human feedback for reinforcement learning, to cite a few avenues. However, the fact that many of these human annotations are inherently subjective is often overlooked. Recent work has demonstrated how ignoring rater subjectivity (typically resulting in rater disagreement) is problematic within specific tasks and for specific subgroups. Generalizable methods to harness rater disagreement and thus understand the socio-cultural leanings of subjective tasks remains an open challenge. In this paper, we propose a comprehensive disagreement analysis framework to measure systematic diversity in perspectives among different rater subgroups, and demonstrate its utility in assessing the extent of group association patterns in two datasets: (1) safety annotations of human-chatbot conversations, and (2) offensiveness annotations of social media posts, both annotated by diverse rater pools across different socio-demographic axes. Our framework (based on disagreement metrics) reveals specific rater groups that have significantly different perspectives than others on certain tasks, and helps identify demographic axes that are crucial to consider in specific task contexts

Paper Type: long

Research Area: Ethics, Bias, and Fairness

Contribution Types: Data analysis

Languages Studied: English