Efficient Training of Multi-task Neural Solver for Combinatorial Optimization

Chenguang Wang, Zhang-Hua Fu, Pinyan Lu, Tianshu Yu

Abstract:

Efficiently training a multi-task neural solver for various combinatorial optimization problems (COPs) has been less studied so far. Naive application of conventional multi-task learning approaches often falls short in delivering a high-quality, unified neural solver. This deficiency primarily stems from the significant computational demands and a lack of adequate consideration for the complexities inherent in COPs. In this paper, we propose a general and efficient training paradigm to deliver a unified combinarotial multi-task neural solver. To this end, we resort to the theoretical loss decomposition for multiple tasks under an encoder-decoder framework, which enables more efficient training via proper bandit task-sampling algorithms through an intra-task influence matrix.
By employing theoretically grounded approximations, our method significantly enhances overall performance, regardless of whether it is within constrained training budgets, across equivalent training epochs, or in terms of generalization capabilities, when compared to conventional training schedules.
On the real-world datasets of TSPLib and CVRPLib, our method also achieved the best results compared to single task learning and multi-task learning approaches.
Additionally, the influence matrix provides empirical evidence supporting common practices in the field of learning to optimize, further substantiating the effectiveness of our approach.
Our code is open-sourced and available at \url{https://github.com/LOGO-CUHKSZ/MTL-COP}.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

We have enhanced our manuscript in response to reviewer feedback by implementing the following improvements:

- We have incorporated confidence intervals corresponding to two standard deviations in Table 2 and Figure 11。
- Figure 2 has been updated to display learning curves that include confidence bands representing two standard deviations with respect to the objective function value.
- We have added the code repository link in the Abstract section to facilitate reproducibility of our work.

Code: https://github.com/LOGO-CUHKSZ/MTL-COP

Assigned Action Editor: ~Xi_Lin2

Submission Number: 3916