BPQP: A Differentiable Convex Optimization Framework for Efficient End-to-End Learning

Jianming Pan, Xiao Yang, Weidong Ma, Weiqing Liu, Lewen Wang, Jiang Bian

Keywords: optimization, differentiable convex layers, machine learning, neural networks

TL;DR: We design an efficient end-to-end convex optimization learning framework to bridge the prediction and decision gap by learning ML models in conjunction with the optimization model to minimize the ultimate decision error.

Abstract:

Real-world decision-making processes often employ a two-stage approach, where a machine learning model first predicts key parameters, followed by a constrained convex optimization model to render final decisions. The machine learning model is typically trained separately to minimize prediction error, which may not necessarily align with the ultimate goal, resulting in potentially suboptimal decisions.
The predict-then-optimize approach offers an end-to-end learning solution to bridge this gap, wherein machine learning models are trained in tandem with the optimization model to minimize the ultimate decision error.
However, practical applications involving large-scale datasets bring about significant challenges due to the inherent need for efficiency to fully realize the potential of the predict-then-optimize approach.
Although recent works have started to focus on predict-then-optimize, they have been limited to small-scale datasets due to low efficiency.
In this paper, we propose BPQP, a differentiable convex optimization framework for efficient end-to-end learning. To address the challenge of efficiency, we initially reformulate the backward pass as a simplified and decoupled quadratic programming problem by exploiting the structural trait of the KKT matrix, followed by solving it using first-order optimization algorithms.
Extensive experiments on both simulated and real-world datasets have been conducted, demonstrating a considerable improvement in terms of efficiency -- at least an order of magnitude faster in overall execution time.
We significantly improve efficiency and highlight the superiority of BPQP compared to baselines, including the traditional two-stage learning approach.

Primary Area: optimization

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 4294