Kernel-based Maximum-of-difference Test for Two-sample Comparison

Dan Pu, Tianyi Zhu, Yao Yan, Wei Lan

TL;DR: We propose a kernel-based Maximum-of-Difference (MOD) test with kernel fusion to overcome MMD’s cancellation issue and enable more powerful two-sample comparisons.

Abstract: Two-sample comparison is a fundamental problem in machine learning, with broad applications such as generative modeling. Although the maximum mean discrepancy (MMD) is widely used, MMD-based tests often exhibit poor or even counterintuitive performance under covariance- and location-shift alternatives, partly due to cancellation effects induced by their sum-of-differences construction. To address this issue, we propose a kernel-based maximum-of-difference (MOD) test, which maximizes the squared discrepancy between within-sample and between-sample average distances, thereby improving sensitivity to subtle distributional differences. We further develop a fused MOD procedure to adaptively combine multiple kernels. Extensive experiments demonstrate clear performance gains over existing MMD-based methods.

Lay Summary: We study how to determine whether two datasets come from the same distribution, a key problem in machine learning with applications such as evaluating generative models. Existing methods can miss subtle differences because they average information across all data points. We propose a new approach, called the maximum-of-difference (MOD) test, that focuses on the largest discrepancies between datasets, making it more sensitive to meaningful changes. We also develop a fused version that combines multiple kernels automatically. Experiments show that the proposed method consistently outperforms existing approaches in detecting distributional differences.

Primary Area: General Machine Learning->Kernel methods

Keywords: Two-sample comparison；Maximum-of-difference test; Kernel fusion

Submission Number: 15636