Character-Aware Convolutional Recurrent Networks with Self-Attention for Emotion Detection on Twitter

Jiangping Huang, Chunli Xiang, Shuwei Yuan, Desen Yuan, Xiaorui Huang

Abstract: Despite myriad efforts in the literature designing neural representation system for emotion detection, very few works consider constructing effective model for apperceiving various emotion intensity on social media because of the informal expression and lack of context. In this paper, we proposed a character-aware convolutional recurrent networks with self-attention for emotion detection on user-generated content. The proposed model contains three parts: the character-level convolutional layer is designed to learn word representation based on character n-grams for capturing subword information and independent on pre-trained word embedding. The recurrent neural networks learn the sequential context information used both forward and backward recurrent neural network. And the self-attention module is used to extract different emotion aspects of the sentence into multiple vector representations. The attention module performs on top of recurrent networks which enables attention to be used in special domain or task when there are no extra inputs. We evaluate the proposed model on two public emotion datasets including both emotion intensity detection and emotion classification. We compare our model with the state-of-the-art methods on these datasets and the experimental results demonstrate that the proposed model outperforms several baselines on most emotion types detection and indicates the effectiveness of the designed model. In addition, the training of the proposed model for these tasks relies exclusively on initialized character vector which can be used for morphologically rich languages with long-tailed frequency distributions or domains with dynamic vocabuaries.