Dimension reduction via score ratio matching

Michael Brennan, Ricardo Baptista, Youssef Marzouk

Keywords: "dimension reduction", "non-Gaussian component analysis", "score matching", "score ratio matching"

TL;DR: We propose a dimension reduction method built around approximating the gradient of the log-ratio between a target and standard-normal density from target samples.

Abstract: We propose a method to detect a low-dimensional subspace where a non-Gaussian target distribution departs from a known reference distribution (e.g., a standard Gaussian). We identify this subspace from gradients of the log-ratio between the target and reference densities, which we call the score ratio. Given only samples from the target distribution, we estimate these gradients via score ratio matching, with a tailored parameterization and a regularization method that expose the low-dimensional structure we seek. We show that our approach outperforms standard score matching for dimension reduction of in-class distributions, and that several benchmark UCI datasets in fact exhibit this type of low dimensionality.

Student Paper: Yes