Deep Learning Quantile Regression for Robustness, Confidence and Planning

Dvir Ben Or, Michael Kolomenkin, Tanya Osokin, Gil Shabat, Hanan Shteingart

Abstract: It is well known that the estimator that minimizes the mean square error loss is the mean. However, the latter is prune to outliers and additionally, in many practical use cases, such as regression confidence bounds for risk, one would like to predict other statistics such as the median, or a certain percentile. In this chapter we will introduce the concept of Quantile Regression, a method based on a unique loss function which allows predicting quantiles of the conditional distribution of the target given the input. Albeit the method is useful and the loss is differential, thus fits modern deep neural network approach, most data science practitioners are not aware for its existence. We hope this chapter will reveal the theory and the know-hows of this methods and will allow building better machine learning systems. In addition, we discuss some recently developed approaches of how to optimize a general loss function under a quantile constraint.