Generalizable Spectral Embedding with an Application to UMAP

TMLR Paper4687 Authors

Abstract: Spectral Embedding (SE) is a popular method for dimensionality reduction, applicable across diverse domains. Nevertheless, its current implementations face three prominent drawbacks which curtail its broader applicability: generalizability (i.e., out-of-sample extension), scalability, and eigenvectors separation. Existing SE implementations often address two of these drawbacks; however, they fall short in addressing the remaining one. In this paper, we introduce $\textit{Sep-SpectralNet}$ (eigenvector-separated SpectralNet), a SE implementation designed to address $\textit{all}$ three limitations. Sep-SpectralNet extends SpectralNet with an efficient post-processing step to achieve eigenvectors separation, while ensuring both generalizability and scalability. This method expands the applicability of SE to a wider range of tasks and can enhance its performance in existing applications. We empirically demonstrate Sep-SpectralNet's ability to consistently approximate and generalize SE, while maintaining scalability. Additionally, we show how Sep-SpectralNet can be leveraged to enable generalizable UMAP visualization. Our code will be publicly available upon acceptance.

Submission Length: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=Lmjqtdy1lU&referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DTMLR%2FAuthors%23your-submissions)

Changes Since Last Submission:

We carefully addressed the Editor's and Reviewers' comments and revised the paper accordingly. The key changes are summarized below:

- Expanded the discussion on the motivation for eigenvector separation and its relevance to downstream applications, with citations to recent related work.
- Revised the experimental analysis to address concerns regarding statistical significance.
- Reorganized the manuscript and updated the method's name to better emphasize the main contribution of the paper: a Spectral Embedding implementation that overcomes all three key limitations of existing methods.
- Increased the number of experimental repetitions, as requested.
- Included a discussion of additional rotation criteria.

Assigned Action Editor: ~Søren_Hauberg1

Submission Number: 4687