Exploring Graph Pre-training for Aspect-based Sentiment Analysis

Xiaoyi Bao, Zhongqing Wang, Guodong Zhou

Submission Type: Regular Long Paper

Submission Track: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Submission Track 2: Natural Language Generation

Keywords: Aspect-based Sentiment Analysis; Generative model; Graph pre-train

Abstract:

Existing studies tend to extract the sentiment elements in a generative manner in order to avoid complex modeling.
Despite their effectiveness, they ignore importance of the relationships between sentiment elements that could be crucial, making the large pre-trained generative models sub-optimal for modeling sentiment knowledge.
Therefore, we introduce two pre-training paradigms to improve the generation model by exploring graph pre-training that targeting to strengthen the model in capturing the elements' relationships.
Specifically, We first employ an Element-level Graph Pre-training paradigm, which is designed to improve the structure awareness of the generative model. Then, we design a Task Decomposition Pre-training paradigm to make the generative model generalizable and robust against various irregular sentiment quadruples.
Extensive experiments show the superiority of our proposed method, validate the correctness of our motivation.

Submission Number: 372