Primal-Dual Neural Algorithmic Reasoning

Yu He, Ellen Vitercik

TL;DR: We propose a general framework for Neural Algorithmic Reasoning to learn the primal-dual (exact and) approximation algorithms.

Abstract: Neural Algorithmic Reasoning (NAR) trains neural networks to simulate classical algorithms, enabling structured and interpretable reasoning over complex data. While prior research has predominantly focused on learning exact algorithms for polynomial-time-solvable problems, extending NAR to harder problems remains an open challenge. In this work, we introduce a general NAR framework grounded in the primal-dual paradigm, a classical method for designing efficient approximation algorithms. By leveraging a bipartite representation between primal and dual variables, we establish an alignment between primal-dual algorithms and Graph Neural Networks. Furthermore, we incorporate optimal solutions from small instances to greatly enhance the model’s reasoning capabilities. Our empirical results demonstrate that our model not only simulates but also outperforms approximation algorithms for multiple tasks, exhibiting robust generalization to larger and out-of-distribution graphs. Moreover, we highlight the framework’s practical utility by integrating it with commercial solvers and applying it to real-world datasets.

Lay Summary: Neural networks have shown promise in simulating simple algorithms like sorting and searching. However, extending this ability to complex problems remains a challenge. Many of the real-world problems are too difficult to solve exactly within a reasonable time. To apply neural networks in practice, they must be able to learn efficient approximations for such problems. In this work, we present a framework that trains neural networks to follow the primal-dual method, a classical approach used to design approximation algorithms for hard problems. We represent these algorithms as graphs, enabling the network to learn reasoning that mirrors the primal-dual method. To improve learning, we incorporate optimal solutions from small problem instances, allowing the model to recognize patterns that apply to larger tasks. Our results show that the model not only replicates but can also outperform the algorithms in some cases. We demonstrate its practical utility by applying it to real-world datasets and enhancing commercial solvers.

Link To Code: https://github.com/dransyhe/pdnar

Primary Area: Deep Learning->Graph Neural Networks

Keywords: Neural algorithmic reasoning, graph neural networks

Submission Number: 7831