Biologically Motivated Algorithms for Propagating Local Target Representations.

Alexander G. Ororbia II, Ankur Mali:

Abstract:

Finding biologically plausible alternatives to back-propagation
of errors is a fundamentally important challenge in artificial
neural network research. In this paper, we propose a learning
algorithm called error-driven Local Representation Alignment
(LRA-E), which has strong connections to predictive coding, a
theory that offers a mechanistic way of describing neurocomputational
machinery. In addition, we propose an improved
variant of Difference Target Propagation, another procedure
that comes from the same family of algorithms as LRA-E.
We compare our procedures to several other biologicallymotivated
algorithms, including two feedback alignment algorithms
and Equilibrium Propagation. In two benchmarks, we
find that both of our proposed algorithms yield stable performance
and strong generalization compared to other competing
back-propagation alternatives when training deeper, highly
nonlinear networks, with LRA-E performing the best overall.