Neural Tangent Kernel Maximum Mean DiscrepancyDownload PDF

21 May 2021, 20:48 (modified: 23 Jan 2022, 19:38)NeurIPS 2021 PosterReaders: Everyone
Keywords: Maximum Mean Discrepancy (MMD), Neural Tangent Kernel (NTK), two-sample test, kernel methods, change-point detection
TL;DR: New kernel MMD statistic computed by neural network stochastic optimization, with theoretical testing power guarantee through NTK approximation.
Abstract: We present a novel neural network Maximum Mean Discrepancy (MMD) statistic by identifying a new connection between neural tangent kernel (NTK) and MMD. This connection enables us to develop a computationally efficient and memory-efficient approach to compute the MMD statistic and perform NTK based two-sample tests towards addressing the long-standing challenge of memory and computational complexity of the MMD statistic, which is essential for online implementation to assimilating new samples. Theoretically, such a connection allows us to understand the NTK test statistic properties, such as the Type-I error and testing power for performing the two-sample test, by adapting existing theories for kernel MMD. Numerical experiments on synthetic and real-world datasets validate the theory and demonstrate the effectiveness of the proposed NTK-MMD statistic.
Supplementary Material: pdf
Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.
5 Replies