A novel Bayesian estimation-based word embedding model for sentiment analysis

Jingyao Tang, Yun Xue, Ziwen Wang, Haoliang Zhao

Keywords: sentiment analysis, sentiment word embeddings, maximum likelihood estimation, Bayesian estimation

TL;DR: We use sentiment probabilities to incorporate the sentiment information of words into the word embeddings and use Bayesian estimation to derive sentiment probabilities.

Abstract: The word embedding models have achieved state-of-the-art results in a variety of natural language processing tasks. Whereas, current word embedding models mainly focus on the rich semantic meanings while are challenged by capturing the sentiment information. For this reason, we propose a novel sentiment word embedding model. In line with the working principle, the parameter estimating method is highlighted. On the task of semantic and sentiment embeddings, the parameters in the proposed model are determined by using both the maximum likelihood estimation and the Bayesian estimation. Experimental results show the proposed model significantly outperforms the baseline methods in sentiment analysis for low-frequency words and sentences. Besides, it is also effective in conventional semantic and sentiment analysis tasks.