DRG2vec: Learning Word Representations from Definition Relational Graph

Xiaobo Shu, Bowen Yu, Zhenyu Zhang, Tingwen Liu

Abstract: Even with larger and larger text data available, encoding linguistic knowledge into word embeddings directly from corpora is still difficult. The most intuitive way to incorporate knowledge into word embeddings is to use external resources, such as the largest datasource for describing words - natural language dictionaries. However, previous methods usually neglect the recursive nature of dictionaries and cannot capture the high-order relation. In this paper, we present DRG2vec, a novel and efficient approach for learning word representations, which exploits the inherent recursiveness of dictionaries by modeling the whole dictionary as a homogeneous graph based on the co-occurrence of entry and word in the definition. Moreover, a tailor-made sampling strategy is introduced to generate word sequences from the definition relational graph, and then, the generated sequences are fed to the Skip-gram model with semantic negative sampling for word representation learning. Extensive experiments on sixteen benchmark datasets show that leveraging the recursive dictionary graph indeed achieves better performance than other state-of-the-art methods, especially exhibits a weighted average improvement of 8.6% in the word similarity task.