FedFwd: Federated Learning without Backpropagation

Seonghwan Park, Dahun Shin, Jinseok Chung, Namhoon Lee

Keywords: federated learning, forward-forward algorithm

TL;DR: We present an approach to federated learning that does not require backpropagation.

Abstract:

In federated learning (FL), clients with limited resources can disrupt the training efficiency.
A potential solution to this problem is to leverage a new learning procedure that does not rely on the computation- and memory-intensive backpropagation algorithm (BP).
This study presents a novel approach to FL called FedFwd that employs a recent BP-free algorithm by Hinton (2022), namely the Forward Forward algorithm, during the local training process.
Unlike previous methods, FedFwd does not require the computation of gradients, and therefore, there is no need to store all intermediate activation values during training.
We conduct various experiments to evaluate FedFwd on standard datasets including MNIST and CIFAR-10, and show that it works competitively to other BP-dependent FL methods.

Submission Number: 88