NMTF-LTM: Towards an Alignment of Semantics for Lifelong Topic Modeling

Zhiqi Lei, Hai Liu, Jiaxing Yan, Yanghui Rao, Qing Li

Abstract: Aiming at mining high quality topics by accumulating and utilizing semantic knowledge for a stream of documents, lifelong topic modeling (LTM) has attracted more and more attentions recently. However, the permutation of topics may change over time, resulting in a http://www.w3.org/1998/Math/MathML" xmlns:xlink="semantic" target="_blank" rel="nofollow">http://www.w3.org/1999/xlink">semantic misalignment between the topic representations of document chunks across the stream. Such a misalignment deteriorates the model performances of various downstream tasks, while it has been overlooked by the existing lifelong topic models. Towards addressing the misalignment of semantics, we formulate LTM as a problem of non-negative matrix tri-factorization (NMTF) and propose a consolidation framework (i.e., NMTF-LTM) to enforce an alignment in a mapped topic space. In addition, a distributed parallel algorithm, namely PNMTF-LTM, is developed to meet the real-time requirement for large-scale stream processing. Empirical results show that our method can not only obtain a superior alignment of semantics without loss of topic quality, but also achieve effective speedup when deployed to a high performance computing cluster.