Is a Small Matrix Eigendecomposition Sufficient for Spectral Clustering?

ICLR 2026 Conference Submission12219 Authors

Keywords: Spectral clustering, Kernel mothods, Distributional kernel

Abstract:

Spectral clustering has been widely used in clustering tasks due to its effectiveness. However, its key step, eigendecomposition of an $n\times n$ matrix, is computationally expensive for large-scale datasets. Recent works have proposed methods to reduce this complexity, such as Nystr\"om method approximation and landmark-based approaches. These methods aim to maintain good clustering quality while performing eigendecomposition on a smaller matrix. The current minimum matrix size for spectral decomposition in spectral clustering is $k\times k$ (where $k$ is the number of clusters). However, no existing algorithms can achieve good clustering performance with only a $k\times k$ matrix eigendecomposition.
In this paper, we propose a novel distribution-based spectral clustering. Our method constructs an $n\times k$ bipartite graph between $n$ data points and $k$ distributions, enabling the eigendecomposition of only a $k\times k$ matrix and preserving clustering quality at the same time. Extensive experiments performed on synthetic and real-world datasets demonstrate the superiority and effectiveness of the proposed method compared to the state-of-the-art algorithms.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 12219