Hyperbolic Embeddings of Supervised Models

Richard Nock, Ehsan Amid, Frank Nielsen, Alexander Soen, Manfred K Warmuth

Keywords: Hyperbolic geometry, supervised model embedding, decision trees, boosting

TL;DR: A full-fledged solution to embed supervised *models* in hyperbolic geometry, and more

Abstract:

Models of hyperbolic geometry have been successfully used in ML for two main tasks: embedding *models* in unsupervised learning (*e.g.* hierarchies) and embedding *data*.
To our knowledge, there are no approaches that provide embeddings for supervised models; even when hyperbolic geometry provides convenient properties for expressing popular hypothesis classes, such as decision trees (and ensembles).
In this paper, we propose a full-fledged solution to the problem in three independent contributions. The first linking the theory of losses for class probability estimation to hyperbolic embeddings in Poincar\'e disk model. The second resolving an issue for a clean, unambiguous embedding of (ensembles of) decision trees in this model. The third showing how to smoothly tweak the Poincar\'e hyperbolic distance to improve its encoding and visualization properties near the border of the disk, a crucial region for our application, while keeping hyperbolicity.
This last step has substantial independent interest as it is grounded in a generalization of Leibniz-Newton's fundamental Theorem of calculus.

Primary Area: Learning theory

Submission Number: 6401