Gaussian Processes with Bayesian Inference of Covariate Couplings

Mattia Rosso, Juho Ylä-Jääski, Zheyang Shen, Markus Heinonen, Maurizio Filippone

Authors that are also TMLR Expert Reviewers: ~Markus_Heinonen1

Abstract: Gaussian processes are powerful probabilistic models that are often coupled with ARD capable of uncovering the importance of individual covariates. We develop covariances characterized by affine transformations of the inputs, formalized via a precision matrix between covariates, which can uncover covariate couplings for enhanced interpretability. We study a range of couplings priors from Wishart to Horseshoe and present fully Bayesian inference of such precision matrices within sparse Gaussian processes. We empirically demonstrate the efficacy and interpretability of this approach.

Certifications: Expert Certification

Submission Length: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=jbg6H7n6Um

Changes Since Last Submission:

Below is a summary of the main revisions:

1. We relocated the subsection on computational complexity of the ACD approach to Section 2.
2. We fixed the typo in Equation (10).
3. We included the missing references suggested by the reviewers.
4. We expanded the discussion on the following points that emerged during our rebuttal, specifically: the use of FITC approximation in the BSGP approach, the rotation-invariant property of the Cholesky factorization, and the kind of approximation we employed for the Horseshoe prior.
5. We included the links to the github repositories containing the code to reproduce the experiments.

Code: https://github.com/mattyred/BayesianSGP_Automatic_Coupling_Determination

Assigned Action Editor: ~Arto_Klami1

Submission Number: 4200