Fréchet Geodesic Boosting

Yidong Zhou, Su I Iao, Hans-Georg Müller

Keywords: geodesic metric space, gradient boosting, non-Euclidean output, Wasserstein space

TL;DR: We propose FGBoost, a gradient boosting framework designed to intrinsically model complex regression relationships with non-Euclidean outputs in geodesic metric spaces.

Abstract: Gradient boosting has become a cornerstone of machine learning, enabling base learners such as decision trees to achieve exceptional predictive performance. While existing algorithms primarily handle scalar or Euclidean outputs, increasingly prevalent complex-structured data, such as distributions, networks, and manifold-valued outputs, present challenges for traditional methods. Such non-Euclidean data lack algebraic structures such as addition, subtraction, or scalar multiplication required by standard gradient boosting frameworks. To address these challenges, we introduce Fréchet geodesic boosting (FGBoost), a novel approach tailored for outputs residing in geodesic metric spaces. FGBoost leverages geodesics as proxies for residuals and constructs ensembles in a way that respects the intrinsic geometry of the output space. Through theoretical analysis, extensive simulations, and real-world applications, we demonstrate the strong performance and adaptability of FGBoost, showcasing its potential for modeling complex data.

Primary Area: General machine learning (supervised, unsupervised, online, active, etc.)

Submission Number: 6125