% !TEX root = ../main.tex

We use Bayesian coreset principles to derive an sparse approximation to the true $\gp$ regression posterior that is learnable via \textit{stochastic} variational inference.

Bayesian coresets search for samples from a smaller data subset that can,
via weighted likelihoods, approximate otherwise hard to compute posterior distributions~\citep{huggins2016coresets, campbell2018bayesian, campbell2019automated, j-jubran2019}.
From an optimization perspective, Bayesian coresets can also be understood as a set of \textit{learnable} (observed or unobserved) points selected to minimize some divergence to a distribution of interest~\citep{ manousakas2020bayesian, manousakas2022black}.

Inspired by such framework,
we posit a coreset-based, variational posterior distribution for $\gp$s (CVGP):
\ie we learn a small subset of \emph{pseudo-inputs} $\XbC=\{\xb_1, \cdots, \xb_M\}$,
and \emph{pseudo-observations} $\ybC=\{y_1, \cdots, y_M\}$,
that if reweighted appropriately with parameters $\betabC=(\beta_{1}, \cdots, \beta_M)$,
approximate the $\gp$ posterior accurately.
Contrary to standard Bayesian coreset methodology,
the coreset tuple $\{\XbC, \ybC \}$ is composed by \emph{learnable pseudo-points} in the input-output data space
---not restricted to the observed empirical data. 

For accurate approximation of the posterior, 
and inspired by ~\citet{titsias2009variational}'s optimal solution,
we ensure that CVGP's posterior obeys the $\gp$ prior-conditional and it's inductive biases (see Section~\ref{ssec:cvtgp_coreset_posterior}).
We learn the CVGP posterior by formulating a variational lower-bound objective that is amenable to its stochastic maximization (see Section~\ref{ssec:cvtgp_lowerbound}).

\subsection{The coreset-based $\gp$ Posterior}

\label{ssec:cvtgp_coreset_posterior}

\input{tex_sections/method_cvtgp_coreset_posterior}

\subsection{CVGP's Variational Lower-Bound}

\label{ssec:cvtgp_lowerbound}

\input{tex_sections/method_cvtgp_lowerbound}

\subsection{A Comparison to Alternatives}

\label{ssec:cvtgp_discuss}

\input{tex_sections/method_cvtgp_discussion}

\subsection{CVGP as Bayesian Coreset Learning}

\label{ssec:cvtgp_inference}

\input{tex_sections/method_inference_and_learning}

