Polar Embedding

Ran Iwamoto, Ryosuke Kohita, Akifumi Wachi

Keywords: natural language processing, word embedding, hierarchical representations, polar coordinates

Abstract: An efficient representation of a hierarchical structure is essential for developing intelligent systems because most real-world objects are arranged in hierarchies. A distributional representation has brought great success in numerous natural language processing tasks, and a hierarchy is also successfully represented with embeddings. Particularly, the latest approaches such as hyperbolic embeddings showed significant performance by representing essential meanings in a hierarchy (generality and similarity of objects) with spatial properties (distance from the origin and difference of angles). To achieve such an effective connection in commonly used Euclidean space, we propose Polar Embedding that learns representations with the polar coordinate system. In polar coordinates, an object is expressed with two independent variables: radius and angles, which allows us to separately optimize their values on the explicit correspondence of generality and similarity of objects in a hierarchy. Also, we introduce an optimization method combining a loss function controlling gradient and iterative uniformization of distributions. We overcome the issue resulting from the characteristics that the conventional squared loss function makes distance apart as much as possible for irrelevant pairs in spite of the fact the angle ranges are limited. Our results on a standard link-prediction task indicate that polar embedding outperforms other embeddings in low-dimensional Euclidean space and competitively performs even with hyperbolic embeddings, which possess a geometric advantage.

One-sentence Summary: Polar embedding represents word hierarchies in polar coordinates.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=DuA6Ma3Zgg