EthiopicEmotion: Multi-label Emotion Dataset with Large Language Models Evaluation

Tadesse Destaw Belay, Olga Kolesnikova

Keywords: Multi-label emotion, intensity, Emotion classification, LLMs

TL;DR: EthiopicEmotion

Abstract: Large Language Models (LLMs) show promising learning and reasoning abilities. Compared to other NLP tasks, multilingual and multi-label emotion evaluation is under-explored in LLMs. In this paper, we present EthiopicEmotion, a multi-label emotion with intensity dataset for four Ethiopian languages, namely Amharic (amh), Afan Oromo (oro), Somali (som), and Tigrinya (tir). We perform extensive experiments with additional English multi-label emotion data from SemEval 2018: Affect in tweets work. Our evaluation includes zero-shot and in-context learning (ICL) from large language models. The result shows that accurate multi-label emotion classification is still insufficient, even for high-resource languages (English), and there is a large gap between the performance of English and low-resource languages.

Submission Number: 16