Generalization of the Change of Variables Formula with Applications to Residual FlowsDownload PDF

02 Jun 2021, 12:40 (edited 09 Jul 2021)INNF+ 2021 posterReaders: Everyone
  • Keywords: Machine Learning, ICML, Change of Variables, Generalization of Flows, Residual Flows, Measure Theory
  • Abstract: Normalizing flows leverage the Change of Variables Formula (CVF) to define flexible density models. Yet, the requirement of smooth transformations (diffeomorphisms) in the CVF poses a significant challenge in the construction of these models. To enlarge the design space of flows, we introduce $\mathcal{L}$-diffeomorphisms as generalized transformations which may violate these requirements on zero Lebesgue-measure sets. This relaxation allows e.g. the use of non-smooth activation functions such as ReLU. Finally, we apply the obtained results to planar, radial, and contractive residual flows.
3 Replies