Spectral Self-supervised Feature Selection

Daniel Segal, Ofir Lindenbaum, Ariel Jaffe

Abstract: Choosing a meaningful subset of features from high-dimensional observations in unsupervised settings can greatly enhance the accuracy of downstream analysis tasks, such as clustering or dimensionality reduction, and provide valuable insights into the sources of heterogeneity in a given dataset. In this paper, we propose a self-supervised graph-based approach for unsupervised feature selection. Our method's core involves computing robust pseudo-labels by applying simple processing steps to the graph Laplacian's eigenvectors. The subset of eigenvectors used for computing pseudo-labels is chosen based on a model stability criterion. We then measure the importance of each feature by training a surrogate model to predict the pseudo-labels from the observations. Our approach is shown to be robust to challenging scenarios, such as the presence of outliers and complex substructures. We demonstrate the effectiveness of our method through experiments on real-world datasets from multiple domains, with a particular emphasis on biological datasets.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission: Improved phrasing, fixed indices and grammar, and added camera ready details such as author details and broader impact statement.

Code: https://github.com/segalon/ssfs

Assigned Action Editor: ~Patrick_Flaherty1

Submission Number: 2965