Riemannian Fuzzy K-Means on Product Manifolds

Jh Yuan, Zhuo Liu, Feiping Nie

Keywords: non-euclidean representation learning, hyperbolic geometry, hyperspherical geometry

Abstract: In this paper, we address an open problem: how to perform fast clustering on product manifolds.} With the increasing interest in non-Euclidean data representations, clustering such data has become an important problem. However, a naive extension of the classic K-Means algorithm to product manifolds requires $\mathcal{O}(\nu \omega)$ time, where $\omega$ is the number of alternating iterations and $\nu $ is the time complexity of each Riemannian optimization. Due to the need for numerous Riemannian optimizations, the naive Riemannian K-Means (NRK) is not suitable for large-scale data. To this end, we propose the Riemannian Fuzzy K-Means (RFK) algorithm for product manifolds, which reduces the time complexity to $\mathcal{O}(\nu )$. Importantly, RFK is not a straightforward extension of K-Means or Fuzzy K-Means to manifolds, it avoids the computation of the Fréchet mean and and achieve a true single-loop optimization. Furthermore, we introduce Radan to accelerate the optimization of RFK. We conduct extensive experiments. RFK and Radan outperform across nearly all metrics in almost every dataset, reaching an impressive level of performance. \textbf{RFK and Radan have been integrated into several non-Euclidean machine learning libraries, such as here.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 4809