Dynamic Task-Specific Factors for Meta-Embedding

Yuqiang Xie, Yue Hu, Luxi Xing, Xiangpeng Wei

Abstract: Meta-embedding is a technology to create a new embedding by combining different existing embeddings, which captures complementary aspects of lexical semantics. The supervised learning of task-specific meta-embedding is a convenient way to make use of accessible pre-trained word embeddings. However, the weights for different word embeddings are hard to calculate. We introduce the dynamic task-specific factors into meta-embedding (DTFME), which are utilized to calculate appropriate weights of different embedding sets without increasing complexity. Then, we evaluate the performance of DTFME on sentence representation tasks. Experiments show that our method outperforms prior works in several benchmark datasets.