Completing the Model Optimization Process by Correcting Patterns of Failure in Regression Tasks

Thomas Bonnier

Keywords: Error pattern correction, Regression tasks, AutoML

TL;DR: We focus on regression tasks and introduce an additional stage to the model optimization process in order to correct error patterns.

Abstract: Model selection and hyper-parameter optimization sometimes prove to be complex and costly processes with unfinished outcomes. In fact, a so-called optimized model can still suffer from patterns of failure when predicting on new data, affecting the generalization error. In this paper, we focus on regression tasks and introduce an additional stage to the model optimization process in order to render it more reliable. This new step aims to correct error patterns when the model makes predictions on unlabeled data. To that end, our method includes two techniques. AutoCorrect Rules leverage the model under/overestimation bias and applies simple rules to adjust predictions. AutoCorrect Model is a supervised approach which exploits different representations to predict residuals in order to revise model predictions. We empirically prove the relevance of our method on the outcome of an AutoML tool using different time budgets, and on a specific optimization case leveraging a pre-trained model for an image regression task.