Practical Integration via Separable Bijective Networks

Christopher M Bender, Patrick Emmanuel, Michael K. Reiter, Junier Oliva

Keywords: integration, flow, likelihood, classification, regression, out of distribution, regularization

Abstract:

Neural networks have enabled learning over examples that contain thousands of dimensions.
However, most of these models are limited to training and evaluating on a finite collection of \textit{points} and do not consider the hypervolume in which the data resides.
Any analysis of the model's local or global behavior is therefore limited to very expensive or imprecise estimators.
We propose to formulate neural networks as a composition of a bijective (flow) network followed by a learnable, separable network.
This construction allows for learning (or assessing) over full hypervolumes with precise estimators at tractable computational cost via integration over the \textit{input space}.
We develop the necessary machinery, propose several practical integrals to use during training, and demonstrate their utility.

One-sentence Summary: We explore a method that enables learning over hypervolumes within the data space.