Distortion-free and GPU-compatible Tree Embeddings in Hyperbolic Space
Keywords: Hyperbolic Geometry, Hyperbolic Tree Embeddings, Representation Learning, Hierarchical Learning
TL;DR: In this paper we propose a method for embedding trees in hyperbolic space by optimizing hyperspherical point separation and using floating point expansion arithmetic for maintaining GPU-compatibility.
Abstract: Embedding tree-like data, from hierarchies to ontologies and taxonomies, forms a well-studied problem for representing knowledge across many domains. Hyperbolic geometry provides a natural solution for embedding trees, with vastly superior performance over Euclidean embeddings. Recent literature has shown that hyperbolic tree embeddings can even be placed on top of neural networks for hierarchical knowledge integration in deep learning settings. For all applications, a faithful embedding of trees is needed, with combinatorial constructions emerging as the most effective direction. This paper identifies and solves two key limitations of existing works. First, the combinatorial construction hinges on finding maximally separated points on a hypersphere, a notoriously difficult problem. Current approaches lead to poor separation, which degrades the quality of the corresponding hyperbolic embedding. As a solution, we propose maximally separated Delaunay tree embeddings (MS-DTE), where during placement, the children of a node are maximally separated through optimization, which directly leads to lower embedding distortion. Second, low distortion requires additional precision. The current approach for increasing precision is to use multiple precision arithmetic, which renders the embeddings useless on GPUs in deep learning settings. We reformulate the combinatorial construction using floating point expansion arithmetic, leading to superior embedding quality while simultaneously retaining their use on accelerated hardware.
Primary Area: learning on graphs and other geometries & topologies
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 4993
Loading