Hyperbolic Neural Networks++Download PDF

Published: 12 Jan 2021, Last Modified: 22 Oct 2023ICLR 2021 PosterReaders: Everyone
Keywords: Hyperbolic Geometry, Poincaré Ball Model, Parameter-Reduced MLR, Geodesic-Aware FC Layer, Convolutional Layer, Attention Mechanism
Abstract: Hyperbolic spaces, which have the capacity to embed tree structures without distortion owing to their exponential volume growth, have recently been applied to machine learning to better capture the hierarchical nature of data. In this study, we generalize the fundamental components of neural networks in a single hyperbolic geometry model, namely, the Poincaré ball model. This novel methodology constructs a multinomial logistic regression, fully-connected layers, convolutional layers, and attention mechanisms under a unified mathematical interpretation, without increasing the parameters. Experiments show the superior parameter efficiency of our methods compared to conventional hyperbolic components, and stability and outperformance over their Euclidean counterparts.
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics
One-sentence Summary: We present novel methods for constructing hyperbolic neural network architectures in the Poincaré ball model, including a parameter-reduced MLR, geodesic-aware FC layers, convolutional layers, and attention mechanisms.
Code: [![github](/images/github_icon.svg) mil-tokyo/hyperbolic_nn_plusplus](https://github.com/mil-tokyo/hyperbolic_nn_plusplus)
Community Implementations: [![CatalyzeX](/images/catalyzex_icon.svg) 3 code implementations](https://www.catalyzex.com/paper/arxiv:2006.08210/code)
13 Replies