ASAP DML: Deep Metric Learning with Alternating Sets of Alternating Proxies

Yeti Z. Gürbüz, Oğul Can, A. Aydın Alatan

Keywords: Deep Metric Learning, Alternating Projections, Generalization

Abstract: Deep metric learning (DML) aims to minimize empirical expected loss of the pairwise intra-/inter- class proximity violations in the embedding image. We relate DML to feasibility problem of finite chance constraints. We show that minimizer of proxy-based DML satisfies certain chance constraints, and that the worst case generalization performance of the proxy-based methods can be characterized by the radius of the smallest ball around a class proxy to cover the entire domain of the corresponding class samples, suggesting multiple proxies per class helps performance. To provide a scalable algorithm as well as exploiting more proxies, we consider the chance constraints implied by the minimizers of proxy-based DML instances and reformulate DML as finding a feasible point in intersection of such constraints, resulting in a problem to be approximately solved by alternating projections. Simply put, we repeatedly train a regularized proxy-based loss and re-initialize the proxies with the embeddings of the deliberately selected new samples. We apply our method with the well-accepted losses and evaluate on four popular benchmark datasets for image retrieval. Outperforming state-of-the-art, the proposed approach consistently improves the performance of the applied losses.

One-sentence Summary: Reformulation of DML with chance constraints suggests a mechanism in which we iteratively re-initialize class proxies and solve the related proxy-based DML with a regularization term relating consecutive problem instances to improve generalization.