Certified Robustness on Visual Graph Matching via Searching Optimal Smoothing Range

Huaqing Shao, Lanjun Wang, Yongwei Wang, Qibing Ren, Junchi Yan

Primary Area: general machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Visual graph matching (GM), certified robustness, randomized smoothing, joint smoothing distribution

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We introduce the first definition of certified robustness for visual graph matching and proposes a novel method, named CR-OSRS. Experiments on different GM solvers and datasets show that our method achieves state-of-the-art robustness certification.

Abstract:

Deep visual graph matching (GM) is a challenging task in combinatorial learning that involves finding a permutation matrix that indicates the correspondence between keypoints from a pair of images and their associated keypoint positions.
Nevertheless, recent empirical studies have demonstrated that visual GM is susceptible to adversarial attacks, which can severely impair the matching quality and jeopardize the reliability of downstream applications.
To the best of our knowledge, certifying robustness for deep visual GM remains an open challenge, which entails addressing two main difficulties: how to handle the paired inputs and the large permutation output space, and how to balance the trade-off between certified robustness and matching performance.

In this paper, we propose a method, Certified Robustness based on Optimal Smoothing Range Search (CR-OSRS), which provides a robustness guarantee for deep visual GM, inspired by the random smoothing technique. Unlike the conventional random smoothing methods that use isotropic Gaussian distributions, we build the smoothed model with a joint Gaussian distribution, which can capture the structural information between keypoints and mitigate the performance degradation caused by smoothing. We design a global optimization algorithm to search the optimal joint Gaussian distribution that helps achieve a larger certified space and higher matching performance. Considering the large permutation output space, we partition the output space based on similarity, which can reduce the computational complexity and certification difficulty arising from the diversity of the output matrix. Furthermore, we apply data augmentation and a similarity-based regularization term to enhance the smoothed model performance during the training phase. Since the certified space we obtain is high-dimensional and multivariable, it is challenging to evaluate directly and quantitatively, so we propose two methods (sampling and marginal radii) to measure it. Experimental results on GM datasets show that our approach achieves state-of-the-art $\ell_{2}$ certified robustness. The source codes will be made publicly available.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8914