An Algorithm to perform Covariance-Adjusted Support Vector Classification in Non-Euclidean Spaces

ICLR 2026 Conference Submission20322 Authors

Keywords: Support Vector Machine, Euclidean Vector Space, Cholesky Decomposition, Mahalanobis Distance, Whitening

TL;DR: We demonstrate limitations of Support Vector Classification (SVC) in Non-Euclidean spaces due to covariance effects, and propose an algorithm to perform population covariance-adjusted SVC using training data.

Abstract: Traditional Support Vector Machine (SVM) classification is carried out by finding the max-margin classifier for the training data that divides the margin space into two equal sub-spaces. This study demonstrates limitations of performing Support Vector Classification in non-Euclidean spaces by establishing that the underlying principle of max-margin classification and Karush Kuhn Tucker (KKT) boundary conditions are valid only in the Euclidean vector spaces, while in non-Euclidean spaces the principle of maximum margin is a function of intra-class data covariance. The study establishes a methodology to perform Support Vector Classification in Non-Euclidean Spaces by incorporating data covariance into the optimization problem using the transformation matrix obtained from Cholesky Decomposition of respective class covariance matrices, and shows that the resulting classifier obtained separates the margin space in ratio of respective class population covariance. The study proposes an algorithm to iteratively estimate the population covariance-adjusted SVM classifier in non-Euclidean space from sample covariance matrices of the training data. The effectiveness of this SVM classification approach is demonstrated by applying the classifier on multiple datasets and comparing the performance with traditional SVM kernels and whitening algorithms. The Cholesky-SVM model shows marked improvement in the accuracy, precision, F1 scores and ROC performance compared to linear and other kernel SVMs.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 20322