Closed-Form Coordinate Ascent Variational Inference for Student-t Process Regression with Student-t Likelihood

Keisuke Onoue, Takatomi Kubo, Kazushi Ikeda

TL;DR: A tractable variational inference framework for Student-t process regression using data augmentation

Abstract:

Combining a Student-t Process (TP) prior with a Student-t likelihood yields a doubly robust regression model whose intractable posterior has prevented its practical use.
We introduce the first tractable variational inference (VI) framework for this model.
Leveraging the Student-t distribution's scale-mixture representation, we design a structured variational family that affords an analytic evidence lower bound.
To overcome the non-conjugacy of this family, which precludes closed-form updates, we devise a novel projection-based optimization:
we find the optimum in a simpler, factorized family and analytically project it back onto our structured one.
The framework is extended to a scalable sparse, stochastic setting.
Empirical results demonstrate strong performance, particularly in the full-batch setting, establishing this robust model as a practical and powerful tool.

Submission Number: 739