GRI: Graph-based Relative Isomorphism of Word Embedding Spaces

Muhammad Asif Ali, Yan HU, Jianbin Qin, Di Wang

Submission Type: Regular Long Paper

Submission Track: Multilinguality and Linguistic Diversity

Submission Track 2: Multilinguality and Linguistic Diversity

Keywords: Attentive graph Convolutions, Isomorphim, Bi-lingual Induction

TL;DR: We propose Attentive Graph Convolutions to Control the relative isomorphism of Monolingual Embeddings for BLI.

Abstract:

Automated construction of bi-lingual dictionaries using monolingual embedding spaces is a
core challenge in machine translation. The end performance of these dictionaries relies on the
geometric similarity of individual spaces, i.e., their degree of isomorphism. Existing attempts
aimed at controlling the relative isomorphism of different spaces fail to incorporate the
impact of lexically different but semantically related words in the training objective. To
address this, we propose GRI that combines the distributional training objectives with attentive
graph convolutions to unanimously consider the impact of lexical variations of semantically
similar words required to define/compute the relative isomorphism of multiple spaces. Exper
imental evaluation shows that GRI outperforms the existing research by improving the average
P@1 by a relative score of upto 63.6%.

Submission Number: 390