Spectral Self-supervised Feature Selection
Abstract: Choosing a meaningful subset of features from high-dimensional observations in unsupervised settings can greatly enhance the accuracy of downstream analysis, such as clustering or dimensionality reduction, and provide valuable insights into the sources of heterogeneity in a given dataset. In this paper, we propose a self-supervised graph-based approach for unsupervised feature selection. Our method's core involves computing robust pseudo-labels by applying simple processing steps to the graph Laplacian's eigenvectors. The subset of eigenvectors used for computing pseudo-labels is chosen based on a model stability criterion. We then measure the importance of each feature by training a surrogate model to predict the pseudo-labels from the observations. Our approach is shown to be robust to challenging scenarios, such as the presence of outliers and complex substructures. We demonstrate the effectiveness of our method through experiments on real-world datasets, showing its robustness across multiple domains, particularly its effectiveness on biological datasets.
Submission Length: Long submission (more than 12 pages of main content)
Changes Since Last Submission: Added new sections to the paper: stability analysis, hyperparameter selection, additional experiments on synthetic data, interpretability, and analysis of computational complexity. The text changes in the updated version are colored in blue.
Assigned Action Editor: ~Patrick_Flaherty1
Submission Number: 2965
Loading