A Max-Min Approach to the Worst-Case Class Separation Problem

Mohammad Mahdi Omati, Prabhu babu, Petre Stoica, Arash Amini

Abstract: In this paper, we propose a novel discriminative feature learning method based on a minorization-maximization framework for min-max (MM4MM) to address the long-standing “worst-case class separation (WCCS)” problem, which, in our design, refers to maximizing the minimum pairwise Chernoff distance between all class pairs in the low-dimensional subspace. The proposed algorithm relies on the relaxation of a semi-orthogonality constraint, which is proven to be tight at every iteration of the algorithm. To solve the worst-case class separation problem, we first introduce the vanilla version of the proposed algorithm, which requires solving a semi-definite program (SDP) at each iteration. We further simplify it to solving a quadratic program by formulating the dual of the surrogate maximization problem. We also then present reformulations of the worst-case class separation problem that enforce sparsity of the dimension-reducing matrix. The proposed algorithms are computationally efficient and are guaranteed to converge to optimal solutions. An important feature of these algorithms is that they do not require any hyperparameter tuning (except for the sparsity case, where a penalty parameter controlling sparsity must be chosen by the user). Experiments on several machine learning datasets demonstrate the effectiveness of the MM4MM approach.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission: The previous submission fully addressed all reviewer comments, and we have reformatted the manuscript to comply with the camera-ready guidelines. In this version, we have limited the main body of the paper to 12 pages (excluding references and appendix) in accordance with the CR requirements. We have also moved the appendix to appear after the references, beginning on a new page.

Assigned Action Editor: ~Ehsan_Amid1

Submission Number: 4394