Even Faster Hyperbolic Random Forests: A Beltrami-Klein Wrapper Approach

TMLR Paper5162 Authors

Abstract: Decision trees and models that use them as primitives are workhorses of machine learning in Euclidean spaces. Recent work has further extended these models to the Lorentz model of hyperbolic space by replacing axis-parallel hyperplanes with homogeneous hyperplanes when partitioning the input space. In this paper, we show how the \hyperdt\ algorithm can be elegantly reexpressed in the Beltrami-Klein model of hyperbolic spaces. This preserves the thresholding operation used in Euclidean decision trees, enabling us to further rewrite \hyperdt as simple pre-- and post-processing steps that form a wrapper around existing tree-based models designed for Euclidean spaces. The wrapper approach unlocks many optimizations already available in Euclidean space models, improving flexibility, speed, and accuracy while offering a simpler, more maintainable, and extensible codebase.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

* Re-did timing benchmarks, introducing 2 new subfigures for Figure 2 (speedup by depth, speedup by dimension)
* Added Figure 4, reflecting the analysis of a baseline written to match the tree-building logic (but not the splitting logic) of HyperDT exactly. Added relevant commentary to Section 5.2.
* Added Table 1, which analyzes the accuracy difference between HyperDT and Fast-HyperDT under Decision Tree and Random Forest variants. Added relevant commentary to Section 5.2.
* Changed Table 2 to explicitly consider ablations of pre- and post-processing steps; moved LightGBM/Oblique Decision Tree results and regression benchmarks to Appendix. Added relevant commentary to Section 5.3.
* Added Wordnet Classification benchmark (Section 5.4)
* Rewrote Section 2, adding background on hyperbolic hyperplanes (Section 2.2).
* Rewrote the Lemma 4.3 proof to improve clarity.
* Added experimental details (machine used for benchmarking; versions; hyperparameters; Gaussian mixtures) to Section 5.1.
* Added discussion of midpoint ablation to the Conclusion.
* Added additional details on hyperbolic geometry in the Appendix.
* Moved LightGBM and Oblique Decision Tree details to the Appendix.
* Fixed typos pointed out by reviewers.

Assigned Action Editor: ~Alessandro_Sperduti1

Submission Number: 5162