Differentiable extensions with rounding guarantees for combinatorial optimization over permutations

Robert R Nerem, Zhishang Luo, Akbar Rafiey, Yusu Wang

Keywords: Combinatorial optimization, Permutations, Birkhoff-von-Neumann decomposition, matching, quadratic assignment

TL;DR: We introduce the Birkhoff Extension, a polynomial-time, almost-everywhere differentiable relaxation of permutation objectives to doubly stochastic matrices, enabling efficient combinatorial optimization.

Abstract:

Continuously extending combinatorial optimization objectives is a powerful technique commonly applied to the optimization of set functions. However, few such methods exist for extending functions on permutations, despite the fact that many combinatorial optimization problems, such as the quadratic assignment problem (QAP) and the traveling salesperson problem (TSP), are inherently optimization over permutations. We present Birkhoff Extension (BE), an almost-everywhere-differentiable continuous polytime-computable extension of any real-valued function on permutations to doubly stochastic matrices. Key to this construction is our introduction of a continuous variant of the well-known Birkhoff decomposition.
Our extension has several nice properties making it appealing for optimization problems.
First, BE provides a rounding guarantee, namely any solution to the extension can be efficiently rounded to a permutation without increasing the function value. Furthermore, an approximate solution in the relaxed case will give rise to an approximate solution in the space of permutations.
Second, using BE, any real-valued optimization objective on permutations can be extended to an almost-everywhere-differentiable objective function over the space of doubly stochastic matrices.
This makes our BE amenable to not only gradient-descent based optimization, but also unsupervised neural combinatorial optimization where training often requires a differentiable loss.
Third, based on the above properties, we present a simple optimization procedure which can be readily combined with existing optimization approaches to offer local improvements (i.e., the quality of the final solution is no worse than the initial solution).
Finally, we also adapt our extension to optimization problems over a class of trees, such as Steiner tree and optimization-based hierarchical clustering.
We present experimental results to verify our theoretical results on several combinatorial optimization problems related to permutations.

Primary Area: Optimization (e.g., convex and non-convex, stochastic, robust)

Submission Number: 13327