Sailing in high-dimensional spaces: Low-dimensional embeddings through angle preservation

Jonas Fischer, Rong Ma

Keywords: data visualization, low-dimensional embeddings, dimensionality reduction

Abstract:

Low-dimensional embeddings (LDEs) of high-dimensional data are ubiquitous in science and engineering. They allow us to quickly understand the main properties of the data, identify outliers and processing errors, and inform the next steps of data analysis.
As such, LDEs have to be *faithful* to the original high-dimensional data, i.e., they should represent the relationships that are encoded in the data, both at a local as well as global scale.
The current generation of LDE approaches focus on reconstructing *local distances* between any pair of samples correctly, often outperforming traditional approaches aiming at all distances.
For these approaches, global relationships are, however, usually strongly distorted, often argued to be an inherent trade-off between local and global structure learning for embeddings. We suggest a new perspective on LDE learning, reconstructing *angles* between data points.
We show that this approach, Mercat, yields good reconstruction across a diverse set of experiments and metrics, and preserve structures well across all scales, outperforming existing methods across datasets and metrics in most cases by a margin.
Compared to existing work, our approach also has a *simple formulation*, facilitating future theoretical analysis and algorithmic improvements.

Primary Area: interpretability and explainable AI

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 7465