Automatic Differentiation in MyiaDownload PDF

28 Oct 2017NIPS 2017 Workshop Autodiff SubmissionReaders: Everyone
Keywords: automatic differentiation, programming languages, machine learning, deep learning
Abstract: Automatic differentiation is an essential feature of machine learning frameworks.However, its implementation in existing frameworks often has limitations. In dataflow programming frameworks such as Theano or TensorFlow the representation used makes supporting higher-order gradients difficult. On the other hand, operator overloading frameworks such as PyTorch are flexible, but do not lend themselves well to optimization. With Myia, we attempt to have the best of both worlds: Building on the work by Pearlmutter and Siskind we implement a first-order gradient operator for a subset of the Python programming language.
4 Replies