KGAT: An Enhanced Graph-Based Model for Text Classification

Xin Wang, Chao Wang, Haiyang Yang, Xingpeng Zhang, Qi Shen, Kan Ji, Yuhong Wu, Huayi Zhan

Abstract: As a fundamental task in natural language processing, text classification, which is to predict the class label of a given text, has been intensively studied. Consequently, a host of techniques have been developed, among which techniques that are based on graph neural network and its variant $$e.g., $$ graph attention network ( $$\textsc {GAT}$$ ) achieved impressive performances, as they show superiority in dealing with complex graph-structured data. Despite effectiveness, most of these techniques suffer from several limitations, $$e.g., $$ incapability in well-capturing correlation among words in a text. In light of these, we propose a comprehensive approach $$\textsc {KGAT}$$ which incorporates multi-head $$\textsc {GAT}$$ with enhanced attention and customized ReadOut operation for text classification. (1) Our approach constructs a text graph $$G_T$$ with edge weights from a text such that both semantic and structural information (with correlation degree) can be well captured. (2) On text graph $$G_T$$ , a novel attention mechanism is incorporated in a multi-head $$\textsc {GAT}$$ for representation learning. (3) Our approach customizes ReadOut operation such that the representation of a text is refined by using a set of influential nodes of $$G_T$$ . Intensive experimental studies on both typical benchmark datasets and a newly created one ( $${\textsf{Sensitive}}$$ ) show that our approach substantially outperforms other baseline methods and yields a promising technique for text classification.