'Slightly disappointing' vs. 'worst sh** ever': tackling cultural differences in negative sentiment expressions in AI-based sentiment analysis

Franziska Sofia Hafner, Lena Hafner, Roberto Corizzo

Abstract: Advertisers, politicians, and social scientists alike have an interest in the current zeitgeist. With people expressing their sentiments in online comments, the Internet has become a great source for ‘reading’ the zeitgeist via artificial intelligence-based sentiment analysis (SA). At this, negative sentiments are of special concern. They can serve as early indicators for events that require action, such as dropping customer satisfaction, discontent amongst potential voters, or a threat of social unrest. However, due to cultural differences in how negative sentiments are expressed, conventional SA methods typically classify such texts with higher accuracy for some ethnicities compared to others. In this paper, we demonstrate this using a large real-world corpus of Google Maps reviews. Across eight ethnic groups, linguistic patterns vary more starkly in negative (1 star) reviews than in neutral (2–4 star) and positive (5 star) reviews. Consequently, ethnicity-blind SA methods ‘struggle’ to classify negative reviews correctly. To mitigate this problem, we propose a novel SA method, based on balanced training and subsequent ethnicity-conscious fine-tuning. Our approach is simultaneously able to mitigate bias and enhance overall model performance. Thus, we hope to contribute to a more equal appreciation of negative sentiments of ethnically diverse customer-, voter-, or research populations, and, consequently, a more nuanced approximation of the overall zeitgeist.