A Markov decision process for variable selection in Branch and bound

Paul STRANG, Zacharie ALES, Côme Bissuel, Safia Kedad-Sidhoum, Olivier Juan, Emmanuel Rachelson

Keywords: Mixed-integer linear programming; Branch and bound; Reinforcement learning; Markov decision process

TL;DR: We unlock the potential for reinforcement learning applications in mixed-integer linear programming by formulating variable selection in Branch and bound as a Markov decision process.

Abstract: Mixed-Integer Linear Programming (MILP) is a powerful framework used to address a wide range of NP-hard combinatorial optimization problems, often solved by Branch and bound (B\&B). A key factor influencing the performance of B\&B solvers is the variable selection heuristic governing branching decisions. Recent contributions have sought to adapt reinforcement learning (RL) algorithms to the B\&B setting to learn optimal branching policies, through Markov Decision Processes (MDP) inspired formulations, and ad hoc convergence theorems and algorithms. In this work, we introduce B\&B MDPs, a principled vanilla MDP formulation for variable selection in B\&B, allowing to leverage a broad range of RL algorithms for the purpose of learning optimal B\&B heuristics. Computational experiments validate our model empirically, as our branching agent outperforms prior state-of-the-art RL agents on four standard MILP benchmarks.

Primary Area: reinforcement learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 11797