Uncertainty Quantification in SVM prediction

TMLR Paper4935 Authors

Abstract: This paper explores SVM models from the lens of uncertainty quantification (UQ), developed for regression and forecasting tasks.} Unlike the Neural Network, the SVM solutions are typically more \textcolor{black}{certain}, stable, sparse, optimal and interpretable. However, there is only limited literature addressing uncertainty quantification (UQ) in SVM-based prediction. \textcolor{black}{We start with providing a comprehensive summary of existing Prediction Interval (PI) estimation and probabilistic forecasting methods developed in the SVM framework. We realize that SVM based PI models lack the sparse solution, a key advantage of SVM solution.} To introduce sparsity in SVM model, we propose the Sparse Support Vector Quantile Regression (SSVQR) model, which constructs PIs and probabilistic forecasts by solving a pair of linear programs. Thereafter, we develop a feature selection algorithm for PI estimation using SSVQR that effectively eliminates a significant number of features while improving PI quality in case of high-dimensional dataset. Additionally, we extend the SVM models in Conformal Regression (CR) setting for obtaining more stable prediction set with finite test set guarantees. Extensive experiments on artificial, real-world benchmark datasets compare the different characteristics of both existing and proposed SVM-based PI estimation methods. \textcolor{black}{ It also highlights the advantages of the feature selection in PI estimation and advantages of SVM based CR model over NN baased CR model.} \textcolor{black}{Furthermore, we compare both, the existing and proposed SVM-based PI estimation models, with modern deep learning models for probabilistic forecasting tasks on three benchmark time-series datasets and find that SVM based PI models obtain better PI coverage and width than modern complex deep learning models.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission: As per the reviewer queries, we have clearly detailed the motivation behind the studying the SVM models from the lens of the UQ in the revised manuscript and also further argues that why the SVM models are less uncertain, more stable and trustworthy than Neural Network (NN). We have modified the contributions and abstract in the revised work for improving the overall comprehension to the reader. We have also modified the Conclusion Section to reflect the impact of our work. Additionally, we have clearly defined our experimental objectives in starting of the Experimental Section and systematically presented the numerical results, including details of the experimental setup, parameter tuning, and baseline models. Furthermore, we have also added a detailed mathematical derivation of the SSVQR optimization problem. We have carefully revised the manuscript to incorporate the reviewers' suggestions and have provided a detailed point-by-point response to all queries and concerns. As part of the supplementary material, we have included a single PDF that first presents our point-by-point responses to the reviewers’ comments, followed by the revised manuscript with all changes highlighted in blue.

Assigned Action Editor: ~Michele_Caprio1

Submission Number: 4935