Learning Topics using Semantic Locality

Ziyi Zhao, Krittaphat Pugdeethosapol, Sheng Lin, Zhe Li, Yanzhi Wang, Qinru Qiu

Abstract:

The topic modeling discovers the latent topic probability of given the text documents. To generate the more meaningful topic that better represents the given document, we proposed a universal method which can be used in the data preprocessing stage. The method consists of three steps. First, it generates the word/word-pair from every single document. Second, it applies a two way parallel TF-IDF algorithm to word/word-pair for semantic filtering. Third, it uses the k-means algorithm to merge the word pairs that have the similar semantic meaning.

Experiments are carried out on the Open Movie Database (OMDb), Reuters Dataset and 20NewsGroup Dataset and use the mean Average Precision score as the evaluation metric. Comparing our results with other state-of-the-art topic models, such as Latent Dirichlet allocation and traditional Restricted Boltzmann Machines. Our proposed data preprocessing can improve the generated topic accuracy by up to 12.99\%. How the number of clusters and the number of word pairs should be adjusted for different type of text document is also discussed.

TL;DR: We proposed a universal method which can be used in the data preprocessing stage to generate the more meaningful topic that better represents the given document