Emerging Topic Detection from Microblog Streams Based on Emerging Pattern Mining

Min Peng, Shuang Ouyang, Jiahui Zhu, Jiajia Huang, Hua Wang, Jianming Yong

Abstract: Emerging topic detection from microblogs has developed into an attractive task because events usually break on social channels. However, due to the features of high noise, short length, fast arriving rate and irregular writing style of microblogs, it has been proven to be a challenge to detect emerging topics from microblog streams early and accurately in a scalable way. Several approaches have been proposed to tackle this problem and have achieved sound performance in some aspects. However, from the point of novelty and scalability, there is still considerable space for improvement. Inspired by the consideration, we propose an emerging topic detection framework based on emerging pattern mining. Via encoding the term novelty into an efficient high utility itemset mining (HUIM) algorithm, a group of emerging patterns which are concise and interpretive representations of topics can be first detected, decreasing the computational cost of the clustering part.