Non-metric multidimensional scaling at scale

Dmitry Kobak

Keywords: dimensionality reduction

Abstract: Multidimensional scaling (MDS) is a method to construct a low-dimensional embedding that approximates pairwise distances in a high-dimensional dataset. MDS exists in several flavors, with metric MDS approximating distances directly, while non-metric MDS additionally optimizing for an arbitrary monotonic transformation of the high-dimensional distances. Most existing MDS implementations have quadratic complexity and do not allow embedding large datasets; some fast stochastic MDS implementations have been recently developed, but only for metric MDS. Here we develop a fast MDS implementation, supporting both metric and non-metric MDS, using stochastic gradient descent in PyTorch. This allows us, for the first time, to construct non-metric MDS embeddings of datasets with sample sizes in tens of thousands. We conduct an empirical study of non-metric MDS using multiple simulated and real-world datasets, including a population genomic and a scRNA-seq dataset, and show that it can strongly outperform metric MDS in terms of global structure preservation.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 24584