Practical Integration via Separable Bijective NetworksDownload PDF

29 Sept 2021, 00:34 (edited 15 Mar 2022)ICLR 2022 PosterReaders: Everyone
  • Keywords: integration, flow, likelihood, classification, regression, out of distribution, regularization
  • Abstract: Neural networks have enabled learning over examples that contain thousands of dimensions. However, most of these models are limited to training and evaluating on a finite collection of \textit{points} and do not consider the hypervolume in which the data resides. Any analysis of the model's local or global behavior is therefore limited to very expensive or imprecise estimators. We propose to formulate neural networks as a composition of a bijective (flow) network followed by a learnable, separable network. This construction allows for learning (or assessing) over full hypervolumes with precise estimators at tractable computational cost via integration over the \textit{input space}. We develop the necessary machinery, propose several practical integrals to use during training, and demonstrate their utility.
  • One-sentence Summary: We explore a method that enables learning over hypervolumes within the data space.
16 Replies