Perspectives in Play: A Multi-Perspective Approach for More Inclusive NLP Systems

Gizem Gezici

Abstract:

In the realm of Natural Language Processing
(NLP), common approaches for handling human
disagreement consist of aggregating annotators’
viewpoints to establish a single ground truth. How-
ever, prior studies show that disregarding individ-
ual opinions can lead to the side-effect of under-
representing minority perspectives, especially in
subjective tasks, where annotators may systemati-
cally disagree because of their preferences. Recog-
nizing that labels reflect the diverse backgrounds,
life experiences, and values of individuals, this
study proposes a new multi-perspective approach
using soft labels to encourage the development
of the next generation of perspective-aware mod-
els—more inclusive and pluralistic. We conduct
an extensive analysis across diverse subjective text
classification tasks including hate speech, irony,
abusive language, and stance detection, to high-
light the importance of capturing human disagree-
ments, often overlooked by traditional aggregation
methods. Results show that the multi-perspective
approach not only better approximates human la-
bel distributions, as measured by Jensen-Shannon
Divergence (JSD), but also achieves superior clas-
sification performance (higher F1-scores), outper-
forming traditional approaches. However, our ap-
proach exhibits lower confidence in tasks like irony
and stance detection, likely due to the inherent sub-
jectivity present in the texts. Lastly, leveraging Ex-
plainable AI (XAI), we explore model uncertainty
and uncover meaningful insights into model pre-
dictions. All implementation details are available
at our github repo.