Decoupled Kernel Neural Processes: Neural-Network-Parameterized Stochastic Processes Using an Explicit Data-driven Kernel
Abstract: Neural Processes (NPs) are a class of stochastic processes parametrized by neural networks. Unlike traditional stochastic processes (e.g., Gaussian processes), which require specifying explicit kernel functions, NPs implicitly learn kernel functions appropriate for a given task through observed data. While this data-driven learning of stochastic processes has been shown to model various types of data, the current NPs’ implicit treatment of the mean and the covariance of the output variables limits its full potential when the underlying distribution of the given data is highly complex. To address this issue, we introduce a new class of neural stochastic processes, Decoupled Kernel Neural Processes (DKNPs), which explicitly learn a separate mean and kernel function to directly model the covariance between output variables in a data-driven manner. By estimating kernel functions with cross-attentive neural networks, DKNPs demonstrate improved uncertainty estimation in terms of conditional likelihood and diversity in generated samples in 1-D and 2-D regression tasks, compared to other concurrent NP variants. Also, maintaining explicit kernel functions, a key component of Gaussian processes, allows the model to reveal a deeper understanding of underlying distributions.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: - Discussion about the computational complexity (Appendix B)
- Discussion about deep kernel learning (Section 5)
- Results of Oracle GP in Table 1
- Minor modifications of Figure 1 and 7
- Minor revision (typos, grammar errors, notations)
Assigned Action Editor: ~Mauricio_A_Álvarez1
Submission Number: 106
Loading