Adaptive Federated Learning Defences via Trust-Aware Deep Q-Networks

Vedant Palit

Keywords: Federated learning, adversarial robustness, robust aggregation, model poisoning, backdoor attacks, reinforcement learning

TL;DR: Trust-aware DQN learns client trust from multi-signal evidence, robustly defending FL against poisoning/backdoors under non-IID and partial observability.

Abstract: Federated learning is vulnerable to poisoning and backdoor attacks under partial observability. We formulate defence as a partially observable sequential decision problem and introduce a trust-aware Deep Q-Network that integrates multi-signal evidence into client trust updates while optimizing a long-horizon robustness–accuracy objective. On CIFAR-10, we (i) establish a baseline showing steadily improving accuracy, (ii) show through a Dirichlet sweep that increased client overlap consistently improves accuracy and reduces ASR with stable detection, and (iii) demonstrate in a signal-budget study that accuracy remains steady while ASR increases and ROC-AUC declines as observability is reduced, which highlights that sequential belief updates mitigate weaker signals. Finally, a comparison with random, linear-Q, and policy gradient controllers confirms that DQN achieves the best robustness–accuracy trade-off.

Primary Area: reinforcement learning

Submission Number: 11166