Generalising sentiment: Leveraging knowledge transfer for multiclass negative emotion mining

ACL ARR 2024 December Submission685 Authors

Abstract: Data annotation is critical yet challenging, particularly for nuanced negative emotions in resource-scarce and niche contexts like hate speech and incel discourse. We systematically explore zero-, few-, and many-shot prompting strategies across three distinct prompting types—base, chain-of-thought (CoT), and in-context learning (ICL)—we uncover the limitations of LLMs in handling nuanced and domain-specific datasets, such as incel discourse. Our findings reveal persistent biases in emotion perception, offering a roadmap to enhance LLM performance in resource-scarce contexts. Furthermore, we introduce and evaluate a hybrid annotation framework, combining LLM-generated annotations with human refinements, which significantly improves accuracy, inter-annotator agreement, and efficiency. This work advances the understanding of NLP generalisation by demonstrating how LLMs can support and complement human annotation efforts, specifically in highly subjective challenging tasks.

Paper Type: Long

Research Area: Computational Social Science and Cultural Analytics

Research Area Keywords: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Contribution Types: Approaches to low-resource settings, Data analysis

Languages Studied: English

Submission Number: 685