Large Language Model Cascades and Persona-Based In-Context Learning for Multilingual Sexism Detection

Lin Tian, Nannan Huang, Xiuzhen Zhang

Abstract: This paper presents an approach for detecting and categorising sexism in social media posts using large language models (LLMs) and ensemble methods. The sEXism Identification in Social neTworks (EXIST) shared task, part of CLEF 2023, consists of three sub-tasks: Sexism Identification, Source Intention, and Sexism Categorisation. We formulate sexism detection in English and Spanish as text classification problems. A distinctive feature of the EXIST datasets is that, in addition to that each social media post is assigned a hard label from majority vote from all annotations, the soft labels – the labels by all annotators of different gender and age profiles – are also included. We propose cascade strategies to leverage LLMs for learning from hard labels. Our hard label-based system Mario is ranked first for the hard label evaluation on both sexism identification (Task 1) and source intention classification (Task 2) of the EXIST 2023 shared task. Our experiments show that more advanced base LLMs (e.g. Llama-3) can further improve the performance of Mario. To learn from soft labels for sexism identification, we further propose fine-grained in-context learning strategies based on personas of different age and gender profiles. Our experiments show that our few-shot persona-based in-context learning strategy leveraging Llama-3 can achieve reasonable performance for soft label prediction for sexism identification, and outperforms previous approaches of directly fine-tuning ensemble of BERT-based models.