Unveiling Neural Combinatorial Optimization Model Representations Through Probing
Keywords: neural combinatorial optimization, vehicle routing problem, probing, representation learning
TL;DR: Understanding what neural combinatorial optimization models learn through probing.
Abstract: Neural combinatorial optimization (NCO) models have achieved remarkable performance, yet their learned underlying representations remain largely unclear. This hinders real-world application, as industrial stakeholders may want a deeper understanding of NCO models before committing resources. In this paper, we make the first step towards interpreting NCO models by investigating embeddings learned by various architectures through three probing tasks. Specifically, we analyze representative and state-of-the-art attention-based models, including AM, POMO, and LEHD, on the representative Traveling Salesman Problem and Capacitated Vehicle Routing Problem. Our findings reveal that NCO models encode linear representations of Euclidean distances between nodes, while also capturing additional knowledge that help avoid making myopic decisions. Furthermore, we show that architectural choices affect the ability of deep models to accurately represent Euclidean distances and to incorporate non-myopic decision-making strategies. We also verify to what extent NCO models understand the feasibility of constraints. Our work represents an initial effort to interpret NCO models, enhance understanding of why certain architectures outperform others, and demonstrate probing as a valuable tool for analyzing their internal mechanisms.
Primary Area: optimization
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 10771
Loading