Learning Unified Distance Metric Across Diverse Data Distributions with Parameter-Efficient Transfer Learning

Sungyeon Kim, Donghyun Kim, Suha Kwak

Abstract: A common practice in metric learning is to train and test an embedding model for each dataset. This dataset-specific approach fails to simulate real-world scenarios that involve multiple heterogeneous distributions of data. In this regard, we explore a new metric learning paradigm, called Uni-fied Metric Learning (UML), which learns a unified dis-tance metric capable of capturing relations across multi-ple data distributions. UML presents new challenges, such as imbalanced data distribution and bias towards dom-inant distributions. These issues cause standard metric learning methods to fail in learning a unified metric. To address these challenges, we propose Parameter-efficient Unified Metric leArning (PUMA), which consists of a pre-trained frozen model and two additional modules, stochas-tic adapter and prompt pool. These modules enable to capture dataset-specific knowledge while avoiding bias to-wards dominant distributions. Additionally, we compile a new unified metric learning benchmark with a total of 8 different datasets. PUMA outperforms the state-of-the-art dataset-specific models while using about 69 times fewer trainable parameters.