Is Compound Aspect-Based Sentiment Analysis Addressed by ChatGPT?

Anonymous

TL;DR: We evaluate the performance of large language models in compound aspect-based sentiment analysis, with our proposed framework ChatABSA.

Abstract: Aspect-based sentiment analysis (ABSA) aims to predict aspect-based elements from the given text, mainly including four elements, i.e., aspect category, sentiment polarity, aspect term, and opinion term. Extracting pair, triple, or quad of elements is defined as compound ABSA. Due to its challenges and practical applications, such a compound scenario has become an emerging topic. Recently, large language models (LLMs), e.g. ChatGPT, present impressive abilities in tackling various human instructions. In this work, we are particularly curious whether ChatGPT still possesses superior performance in handling compound ABSA tasks. To assess the performance of ChatGPT, we design a novel framework, called ChatABSA. Concretely, we design two strategies: constrained prompts, to automatically organize the returned predictions; post-processing, to better evaluate the capability of ChatGPT in recognition of implicit information. The overall evaluation involves 5 compound ABSA tasks and 8 publicly available datasets. We compare ChatGPT with few-shot supervised baselines and fully supervised baselines, including corresponding state-of-the-art (SOTA) models on each task. Experimental results show that ChatABSA exhibits excellent aspect-based sentiment analysis capabilities and overwhelmingly beats few-shot supervised methods under the same few-shot settings. Surprisingly, it can even outperform fully supervised methods in some cases. However, in most cases, it underperforms fully supervised methods, and there is still a huge gap between its performance and the SOTA method. Moreover, we also conduct a series of correlation analyses to gain a deeper understanding of its sentiment analysis capabilities.

Paper Type: long

Research Area: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Contribution Types: NLP engineering experiment

Languages Studied: English