Comparison of two gradient computation methods in Python

Sri Hari Krishna Narayanan, Paul Hovland, Kshitij Kulshreshtha, Devashri Nagarkar, Kaitlyn MacIntyre, Riley Wagner, Deqing Fu

Abstract: Gradient based optimization and machine learning applications require the computation of derivatives. For example, artificial neural networks (ANNs), a widely used learning system, use backpropagation to calculate the error contribution of each neuron after a batch of data is processed. Languages such as Python and R are are popular for machine learning. Therefore, there has been interest in the last few years to build tools for Python and R to compute derivatives.

TL;DR: Compares the performance and features of two automatic differentiation tools, autograd and ADOL-C

Keywords: Automatic Differentiation, Python, autograd, adolc, performance comparison