Regularized Maximum Mean Discrepancy for Variable Selection

Junfeng Huo, Bingyao Huang, Yanyan Liu, Liuhua Peng

Keywords: Variable selection, Maximum mean discrepancy, Two-sample tests, Binary classification

TL;DR: We propose variables selection method based on the maximum mean discrepancy (MMD), which can effectively screen important variables that cause differences in distributions between two samples.

Abstract:

In this paper, we propose a variable selection method based on maximum mean discrepancy (MMD) to effectively identify important variables that contribute to distributional differences between two samples. We begin by assigning weights to each variable and then optimizing these weights within a regularized MMD framework. The optimized weights serve as an importance measure for each variable and can be leveraged for variable selection. Additionally, using the optimized weights, we design two algorithms aimed at enhancing test power and improving classification accuracy for two-sample tests and classification problems. Our method is model-free and makes no assumptions about the underlying structure of the data. Moreover, we propose an acceleration method to improve computational efficiency.
We also provide theoretical guarantees, including the consistency of the estimated weights and the convergence of our acceleration algorithms. Through numerical simulations and real-world datasets, we validate the effectiveness of the proposed method.

Primary Area: other topics in machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 6394