Towards One Model for Classical Dimensionality Reduction: A Probabilistic Perspective on UMAP and t-SNE
Keywords: probabilistic interpretation, dimensionality reduction, gaussian process, pca
TL;DR: We show that UMAP and t-SNE-like algorithms can be explained as maximum likelihood algorithms in a wishart model similar to PCA but with non-linearity.
Abstract: This paper shows that dimensionality reduction methods such as UMAP and t-SNE, can be approximately recast as MAP inference methods corresponding to a model introduced in ProbDR, that describes the graph Laplacian (an estimate for the precision/inverse covariance) matrix using a Wishart distribution, with a mean given by a non-linear covariance function evaluated on the latents. This interpretation offers deeper theoretical and semantic insights into such algorithms, and forging a connection to Gaussian process latent variable models by showing that well-known kernels can be used to describe covariances implied by graph Laplacians. We also introduce tools with which similar dimensionality reduction methods can be studied.
Submission Number: 1
Loading