Sp-R-IP: A Decision-Focused Learning Strategy for Linear Programs that Avoids Overfitting

Ruben Smets, Mihaly Dolanyi, Jean-François Toubeau, Kenneth Bruninx, Erik Delarue

Primary Area: optimization

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Decision-focused learning, interior point optimization, neural networks, linear programs

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We introduce an interior point optimization method for an early stopping-like approach to avoid overfitting in decision-focused learning with linear downstream optimization programs.

Abstract: For forecast-informed linear optimization problems, neural networks have shown to be effective tools for achieving robust out-of-sample performance. Various decision-focused learning paradigms have further refined those outcomes by integrating the downstream decision problem in the training pipeline. One of these strategies involves using a convex surrogate of the regret loss function to train the forecaster, called the SPO+ loss function. It allows for the training problem to be reformulated as a linear optimization program. However, this strategy has only been applied to linear forecasters, and is prone to overfitting. In this paper, we propose an extension of the SPO+ reformulation framework that solves the forecaster training procedure using an interior-point optimization method, and tracks the validation regret of intermediate results obtained for different weights of the barrier term. Additionally, we extend the reformulation framework to include the possibility of neural network forecasters with non-linear activation functions. On a real-life experiment of maximizing storage profits in a day-ahead electricity market using actual price data, we show that the proposed methodology effectively solves the problem of overfitting, and that it can outperform other decision-focused benchmarks including training the forecaster with implicit differentiation.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 4549