Curriculum metric learning for robust image retrieval

Michal Kucer, Diane Oyen

Primary Area: metric learning, kernel learning, and sparse coding

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: adversarial robustness, robustness prior

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: Robust features significantly improve ranking defenses

Abstract: Deep Metric Learning (DML) is a widely used paradigm for learning data representations used for retrieval, where the goal is to retrieve a set of items that are relevant to a query sample. Similar to other deep learning approaches, DML is vulnerable to various forms of adversarial ranking attacks, which change the retrieval ranking through adversarial perturbations. Current DML defenses initialize their models with pretrained ImageNet weights (as is standard), though we hypothesize this is sub-optimal. Deep models optimized to solve the robust optimization framework are trained to be invariant to a set of perturbations $\Delta$, which we hypothesize is a useful, if not necessary, starting point for robust retrieval. Learning approaches for robust retrieval representations must accomplish two goals: (a) learn semantically meaningful representations, and (b) learn robust representations resistant to ranking attacks. We propose the use of curriculum learning for robust retrieval by decomposing the learning process into two steps: (1) learn robust features followed by (2) robust metric learning to learn semantic features for accurate retrieval. In this work, we demonstrate that imposing robust optimization as a feature prior is critical for learning robust retrieval representations. We show that robust representations learned by robust models possess a certain degree of robustness against ranking attacks. Furthermore, by initialing adversarial ranking defenses with robust weights, we significantly improve model's recall on benign examples and their robustness against adversarial ranking attacks

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3005