In-depth: Marginalisation - Getting Predictions for chosen points¶
A Gaussian Process (GP) represents infinitely many possible function values. But in practice, we only care about predictions at a finite number of test points (Let's say forecasting the weather for the next 7 days). Marginalisation is the operation that lets us “zoom in” on exactly those points, while ignoring the rest.
In practise that means that for a new test input $x_*$, the GP gives us a Gaussian prediction with
mean = prior guess + correction from the data
$$ m_{\vert y}(x_*) = m(x_*) + k(x_*,X)\,[K+\sigma_n^2 I]^{-1}(y-m). $$
variance = prior uncertainty – reduction from the data
$$ k_{\vert y}(x_*,x_*) = k(x_*,x_*) - k(x_*,X)\,[K+\sigma_n^2 I]^{-1}k(X,x_*). $$
How to read the animation
- Forecast points ($x_*$) are shown as blue stars and are the specific future times where we want predictions
- As new observations arrive we see how the predicted mean (blue line) and the uncertainty bands (shaded regions) update
- The posterior covariance heatmap (right panel) shows the uncertainty for each prediction after seeing data (lighter colors = more confident predictions)
Notice that without data, our uncertainty for the prediction is large and the prediction equals our prior guess. As we observe more data, we become more confident and our prediction follows the data more closely.