The Differentiable CurryDownload PDF

20 Sep 2019 (modified: 20 Sep 2019)NeurIPS 2019 Workshop Program Transformations SubmissionReaders: Everyone
  • Keywords: automatic differentiation, higher-order programs
  • Abstract: We revisit the automatic differentiation (AD) of programs that contain higher-order functions, in a statically typed setting. Our presentation builds on a recent formulation of AD based on categorical combinators, and shows how that formulation can be extended to higher-order functions via two different notions of differentiable currying and evaluation. We present these alternative implementations, and justify their correctness by means of showing that AD yields equivalent back-propagators for forward-equivalent programs, even in the presence of higher-order features. Higher-order functions complicate the definition of equivalence for back-propagators compared to simpler forms of program equivalence.
5 Replies