Are skip connections necessary for biologically plausible learning rules?

Daniel Jiwoong Im, Rutuja Patil, Kristin Branson

Keywords: Credit Assignment, Biologically plausible learning rule, skip connections

Abstract: Recognizing that backpropagation has been the workhorse of deep learning, it is time to explore other alternative learning methods. Several biologically motivated learning rules have been introduced, such as random feedback alignment and difference target propagation. However, none of these methods have produced competitive performance against backpropagation. In this paper, we show that biologically motivated learning rules with skip connections between intermediate layers can perform as well as backpropagation on the MNIST dataset and is robust to various sets of hyper-parameters.