Optimal Multiple Transport with Applications to Visual Matching, Model Fusion and Beyond
Primary Area: general machine learning (i.e., none of the above)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Optimal Transport; Sinkhorn Algorithm; Cycle-Consistency; Visual Matching; Model Fusion
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: Optimal transport (OT) has wide applications including machine learning. It concerns finding the optimal mapping for Monge OT (or coupling for Kantorovich OT) between two probability measures. This paper generalizes the classic pairwise OT to the so-called Optimal Multiple Transportation (OMT) accepting more than two probability measures as input. We formulate the problem as minimizing the transportation costs between each pair of distributions and meanwhile requiring cycle-consistency of transportation among probability measures. In particular, we present both the Monge and Kantorovich formulations of OMT and obtain the approximate solution with added entropic and cycle-consistency regularization, for which an iterative Sinkhorn-based algorithm (ROMT-Sinkhorn) is proposed. We empirically show the superiority of our approach on two popular tasks: visual multi-point matching (MPM) and multi-model fusion (MMF). In MPM, our OMT solver directly utilizes the cosine distance between learned features of points obtained from off-the-shelf graph matching neural networks as the pairwise cost. We leverage the ROMT-Sinkhorn algorithm to learn multiple matchings. For MMF, we focus on the problem of fusing three models and employ ROMT-Sinkhorn instead of the Sinkhorn algorithm to learn the alignment between layers. Both tasks achieve competitive results with ROMT-Sinkhorn. Furthermore, we showcase the potential of our approach in addressing the travel salesman problem (TSP) by searching for the optimal path on the probability matrix instead of the distance matrix. Source code will be made publicly available.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 4957
Loading