FORGE: Foundational Optimization Representations from Graph Embeddings

Zohair Shafi, Serdar Kadioglu

Keywords: graph machine learning, combinatorial optimization

TL;DR: foundational model for MIP instances across problem types and difficulties using vector quantized graph auto-encoders

Abstract: Combinatorial optimization problems are ubiquitous in science and engineering, yet learning-based approaches to accelerate their solution often require solving a large number of hard-to-solve optimization instances to collect training data, incurring significant computational overhead. Existing methods require training dedicated models for each problem distribution for each downstream task, limiting their scalability and generalization. In this work, we introduce FORGE, a method of pre-training a vector-quantized graph autoencoder on a large and diverse corpus of mixed-integer programming (MIP) instances in an unsupervised fashion without dependency on their solution. The vector quantization creates discrete code assignments that act as a vocabulary to represent MIP instances. We evaluate FORGE in both supervised and unsupervised settings. For the unsupervised setting, we show that FORGE embeddings effectively differentiate and cluster unseen instances. For the supervised setting, we fine-tune FORGE and show that a single model predicts both the variables for warm-starts and integrality gaps for cut-generation across multiple problem type distributions. Both predictions help improve performance of a SOTA, commercial optimization solver. Finally, we release our code and pre-trained FORGE weights to encourage further research and practical use of instance-level MIP embeddings.

Submission Number: 26