Volume-Aware Distance for Robust Similarity Learning

Shuo Chen, Chen Gong, Jun Li, Jian Yang

TL;DR: A new volume-aware distance for the better similarity learning

Abstract: Measuring the similarity between data points plays a vital role in lots of popular representation learning tasks such as metric learning and contrastive learning. Most existing approaches utilize point-level distances to learn the point-to-point similarity between pairwise instances. However, since the finite number of training data points cannot fully cover the whole sample space consisting of an infinite number of points, the generalizability of the learned distance is usually limited by the sample size. In this paper, we thus extend the conventional form of data point to the new form of data ball with a predictable volume, so that we can naturally generalize the existing point-level distance to a new volume-aware distance (VAD) which measures the field-to-field geometric similarity. The learned VAD not only takes into account the relationship between observed instances but also uncovers the similarity among those unsampled neighbors surrounding the training data. This practice significantly enriches the coverage of sample space and thus improves the model generalizability. Theoretically, we prove that VAD tightens the error bound of traditional similarity learning and preserves crucial topological properties. Experiments on multi-domain data demonstrate the superiority of VAD over existing approaches in both supervised and unsupervised tasks.

Lay Summary: In many machine learning tasks like image classification or data retrieval, measuring how similar data points are is crucial. Traditional methods focus on distances between individual data points, but this has a limitation: real-world data spaces are vast, and we can’t sample every possible point, so models often struggle with new, unseen data. This paper introduces a new approach that treats each data point not as a single dot but as a "data ball" with a predictable volume, like expanding a point into a small region. By calculating the geometric similarity between these regions (instead of just points), the model can better capture relationships between both observed and unobserved data around them. This improves how well the model generalizes to new data, making its decisions more robust and consistent across different scenarios. Theoretical analysis shows this method tightens error bounds and preserves key data structures, while experiments on various datasets—including images, text, and graphs—demonstrate that it outperforms existing techniques in both supervised and unsupervised learning tasks. Essentially, by thinking in terms of data regions rather than isolated points, we create more reliable and adaptable machine learning models.

Primary Area: General Machine Learning->Representation Learning

Keywords: similarity learning, metric learning, contrastive learning, volume-aware, theoretical analysis

Submission Number: 5429