Distributionally Robust Regularization of Sparse Integer Programming Trained Learning Models

Sanjeeb Dash, Soumyadip Ghosh, Joao Goncalves, Mark S. Squillante

Keywords: distributionally robust optimization, mixed integer programming, explainable machine learning, rule-based inference

TL;DR: introduces theory on DRO regularization of mixed integer programs; shows how to apply using sparse ruleset binary classification problem

Abstract: Building explainable machine learning models is crucial for human users to be able to interpret the proposed statistical relationship obtained from the training data. Mixed integer optimization formulations are often used to train such models with explicit sparsity constraints, aiming to hit the right trade off between sparsity and prediction accuracy. Existing methods to find the right choice of sparsity -- e.g., via cross-validation -- are computationally expensive. For convex model training formulations, recent advances in distributionally robust optimization (DRO) provide strong generalization while sidestepping this computational burden. We describe an extension of such regularization via DRO to mixed integer sparse programs, providing statistical guarantees as a function of an associated sparsity parameter of the formulation. We illustrate the use of this approach in the case of building explainable binary classification models using sets of feature value rules.

Submission Number: 70