Spectral Self-supervised Feature Selection

Daniel Segal, Ofir Lindenbaum, Ariel Jaffe

Keywords: Feature Selection, Unsupervised Feature Selection, Spectral Methods, Graph Laplacian

TL;DR: We introduce a self-supervised graph-based method for unsupervised feature selection, leveraging the graph Laplacian's eigenvectors for robust pseudo-label generation and using a stability criterion to select meaningful features.

Abstract:

Selecting a meaningful subset of features from high-dimensional observations in unsupervised settings can significantly improve the accuracy of downstream analysis tasks such as clustering or dimensionality reduction and provide insight into the sources of heterogeneity in a given dataset.
In this paper, we derive a self-supervised graph-based approach for unsupervised feature selection. The core of our method is the robust computation of pseudo-labels by applying simple processing steps to the graph Laplacian's eigenvectors. The subset of eigenvectors used for computing pseudo-labels is chosen according to a model stability criterion. The importance of each feature is then measured by training a surrogate model to predict the pseudo-labels from the observations.
We show that our method is robust to challenging scenarios, such as the existence of outliers and complex substructures.
Our approach's efficacy is demonstrated through experiments on real-world datasets, showing its robustness across multiple domains and particular effectiveness on biological datasets.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5151