On the inability of Gaussian process regression to optimally learn compositional functionsDownload PDF

Published: 31 Oct 2022, Last Modified: 04 Jan 2023NeurIPS 2022 AcceptReaders: Everyone
Keywords: Gaussian processes, Bayesian nonparametrics, posterior contraction, minimax estimation, large-sample asymptotics
TL;DR: We prove information-theoretic lower bounds for convergence rates of Gaussian processes when the true function has a compositional structure.
Abstract: We rigorously prove that deep Gaussian process priors can outperform Gaussian process priors if the target function has a compositional structure. To this end, we study information-theoretic lower bounds for posterior contraction rates for Gaussian process regression in a continuous regression model. We show that if the true function is a generalized additive function, then the posterior based on any mean-zero Gaussian process can only recover the truth at a rate that is strictly slower than the minimax rate by a factor that is polynomially suboptimal in the sample size $n$.
Supplementary Material: pdf
8 Replies