Exploiting Inferential Structure in Neural Processes

Dharmesh Tailor, Mohammad Emtiyaz Khan, Eric Nalisnick

Keywords: neural processes, structured inference

TL;DR: We incorporate structured inference networks into neural processes.

Abstract: Neural processes (NPs) can be extremely fast at test time, but their training requires a wide range of context sets to generalize well. We propose to address this issue by incorporating the structure of graphical models into NPs. This leads to aggregation strategies in which context points are appropriately weighted, generalizing a recent proposal by Volpp et al., [2020]. The weighting further reveals an interpretation of each point, which we refer to as the neural sufficient statistics. It is expected that by exploiting information in structured priors, the data inefficiency of NPs can be alleviated.

Community Implementations: [ 3 code implementations](https://www.catalyzex.com/paper/arxiv:2306.15169/code)