Most likely heteroscedastic Gaussian process via kernel smoothing

Ghifari Adam Faza, Nasrulloh R. B. S. Loka, Keivan Shariatmadar, Hans Hallez, David Moens

Abstract: Highlights•Heteroscedastic modelling using most likely heteroscedastic Gaussian process (MLHGP) and its variations requires training two Gaussian processes (GPs) to model the main function and the residual noise. Which, in this case, can be expensive when we need to fit the data multiple times, such as in Bayesian optimisation settings. To tackle the problem, we introduce the kernel smoothing approximation variant of the MLHGP.•In this paper we relax the procedure of the typical MLHGP that uses two GPs by substituting the noise approximator GP with a kernel smoothing function that exhibits the same value of the length scale parameter as the main GP. Thus, fitting the noise approximator becomes unnecessary.•This approach reduces the computational complexity from O(2N3) to O(N3+N2). Which translates to 2× speed-up during training in our test cases.•We also found that our method offers an advantage over the existing method in the Bayesian optimisation case with a low number of initial data.

External IDs:dblp:journals/kbs/FazaLSHM25