TDAM: A topic-dependent attention model for sentiment analysis

Gabriele Pergola, Lin Gui, Yulan He

Abstract: Highlights • A new recurrent unit derives text representations by an internal attention mechanism. • Internal attention is applied to discover text patterns by means of topic embeddings. • Patterns generated under multi-task supervision generates polarity-bearing topics. • The proposed hierarchical model benefits sentiment classification and topic extraction. Abstract We propose a topic-dependent attention model for sentiment classification and topic extraction. Our model assumes that a global topic embedding is shared across documents and employs an attention mechanism to derive local topic embedding for words and sentences. These are subsequently incorporated in a modified Gated Recurrent Unit (GRU) for sentiment classification and extraction of topics bearing different sentiment polarities. Those topics emerge from the words’ local topic embeddings learned by the internal attention of the GRU cells in the context of a multi-task learning framework. In this paper, we present the hierarchical architecture, the new GRU unit and the experiments conducted on users’ reviews which demonstrate classification performance on a par with the state-of-the-art methodologies for sentiment classification and topic coherence outperforming the current approaches for supervised topic extraction. In addition, our model is able to extract coherent aspect-sentiment clusters despite using no aspect-level annotations for training.