A Two-Stage Exact Algorithm for Optimization of Neural Network Ensemble

Keliang Wang, Leonardo Lozano, David Bergman, Carlos Cardonha

Abstract: We study optimization problems where the objective function is modeled through feedforward neural networks. Recent literature has explored the use of a single neural network to model either uncertain or complex elements within an objective function. However, it is well known that ensembles can produce more accurate and more stable predictions than single neural network. We therefore study how neural network ensemble can be incorporated within an objective function, and propose a two-stage optimization algorithm for solving the ensuing optimization problem. Preliminary computational results applied to a global optimization problem and a real-world data set show that the two-stage model greatly outperforms a standard adaptation of previously proposed MIP formulations of single neural network embedded optimization models.