Relationship constraint deep metric learning

Yanbing Zhang, Ting Xiao, Zhe Wang, Xinru Wang, Wenyi Feng, Zhiling Fu, Hai Yang

Abstract: Deep metric learning (DML) models aim to learn semantically meaningful representations in which similar samples are pulled together and dissimilar samples are pushed apart. However, the classification effect is limited due to the high time complexity of previous models and their poor performance in extracting data relationships. This paper presents a novel relationship constraint deep metric learning (RCDML) approach, including proxy relationship constraint (PRC) and sample relationship constraint (SRC) for inter-class separability and intra-class compactness, to solve the above problems and improve the classification effect. The PRC combines the proxy-to-proxy relationship loss term with the proxy-to-sample relationship loss function to maximize the proxy features, hence enhancing inter-class separability by decreasing proxy similarity. Additionally, the SRC combines the sample-to-sample relationship loss term with the proxy-to-sample relationship loss function to maximize the sample features, which promotes intra-class compactness by increasing the similarity between the most different samples of the same class. Unlike existing proxy-based and pair-based methods, the relationship constraint framework uses a diverse range of proxy and sample data relationships. In addition, the proxy correction (PC) module is used to optimize the proxy. Extensive tests conducted on the widely popular CUB-200-2011, CARS-196, and SOP datasets show that the framework is effective and attains state-of-the-art performance.