Distance-informed Neural Processes

Aishwarya Venkataramanan, Joachim Denzler

Keywords: neural processes, Bi-Lipschitz regularization, Uncertainty estimation, uncertainty calibration, Out-of-distribution (OOD) detection, gaussian processes, stochastic processes

TL;DR: We introduce Distance-informed Neural Processes (DNP), a Neural Process variant with bi-Lipschitz regularization that preserves input geometry. DNP yields better-calibrated uncertainty & stronger OOD detection across regression + classification.

Abstract: We propose the Distance-informed Neural Process (DNP), a novel variant of Neural Processes that improves uncertainty estimation by combining global and distance-aware local latent structures. Standard Neural Processes (NPs) often rely on a global latent variable and struggle with uncertainty calibration and capturing local data dependencies. DNP addresses these limitations by introducing a global latent variable to model task-level variations and a local latent variable to capture input similarity within a distance-preserving latent space. This is achieved through bi-Lipschitz regularization, which bounds distortions in input relationships and encourages the preservation of relative distances in the latent space. This modeling approach allows DNP to produce better-calibrated uncertainty estimates and more effectively distinguish in- from out-of-distribution data. Empirical results demonstrate that DNP achieves strong predictive performance and improved uncertainty calibration across regression and classification tasks.

Primary Area: Probabilistic methods (e.g., variational inference, causal inference, Gaussian processes)

Submission Number: 13225