Wavelet Conditional Neural Processes

Junyu Xuan, Mengjing Wu

Abstract: Conditional neural processes (CNPs) are a new family of stochastic processes defined by deep neural networks, characterized by the necessary properties of marginal consistency and exchangeability. Thanks to their generalization capabilities across tasks, popular applications of CNPs include meta-learning and multi-task learning. The existing CNPs map a context set to a vector or function space where all samples are considered homogeneously, which limits their representational power. In this paper, we introduce a Wavelet Conditional Neural Process (WaveCNP) as a new member of the CNP family, based on wavelet transform theory. We propose mapping the context set into a nested multiresolution function space sequence rather than a singular space, achieved through the efficient and adaptive discrete wavelet transform. We demonstrate that our WaveCNP can outperform existing CNPs in terms of conditional predictive distribution modeling and multiresolution prediction.

Keywords: Neural Processes, Bayesian deep learning

Submission Number: 4