Sabiá in Action: An Investigation of its Abilities in Aspect-Based Sentiment Analysis, Hate Speech Detection, Irony Detection, and Question-Answering

Julia da Rocha Junqueira, Émerson Lopes, Claudio Luis Da S. M., Felix L. V. da Silva, Eduarda Abreu Carvalho, Larissa A. de Freitas, Ulisses Brisolara Corrêa

Abstract: This research investigates the efficacy of the versatile Sabiá-7B model, employed to decipher the complexities of Portuguese language across various tasks. Leveraging state-of-the-art architecture, the model extends LLaMA-7B pre-training to represent the Portuguese language better. This study focuses on evaluating Sabiá-7B’s performance in Aspect-Based Sentiment Analysis (ABSA), Hate Speech Detection (HS), Irony Detection (ID), and Question-Answering (QA) tasks using a few-shot approach. Employing the few-shot method and prompt engineering throughout task executions, our research revealed that Sabiá-7B exhibits notable proficiency, mainly when provided with ample examples during few-shot extraction. However, particular challenges emerged, especially in QA tasks, where the model displayed limitations in generating precise answers compared to expected exact responses. This limitation resulted in the inclusion of extraneous words, potentially classified as irrelevant, impeding the accurate identification of an exact match. Our investigation sheds light on the strengths and potential limitations of Sabiá-7B in various NLP domains. As AI capabilities continue to advance, understanding these intricacies becomes essential for practical applications and the field’s ongoing development.