FL-GAP: GRAPH-BASED ADAPTIVE PERSONALIZA- TION FOR FEDERATED DEEPFAKE DETECTION

Pai Chet Ng, Konstantinos N. Plataniotis, Shanhong Liu, De Wen Soh

Keywords: federated learning, Personalization, Deepfake, Misinformation

TL;DR: a novel framework designed to address the dual challenges of personalization and communication efficiency in federated deepfake detection.

Abstract:

Modern deepfake detection models degrade sharply when faced with unseen gen-
erative techniques or cross-domain shifts, a challenge further exacerbated in Fed-
erated Learning (FL) by heterogeneous client data. Standard FL methods (e.g.,
FedAvg) converge poorly under such conditions, while existing personalized FL
approaches often assume uniform similarity or rely on overly simplistic strategies
that fail to capture nuanced feature shifts. We introduce FL-GAP, a framework
for Federated Learning with Graph-based Adaptive Personalization that system-
atically adapts to both client heterogeneity and generator shift. FL-GAP combines
three components: (1) Adaptive Layer Freezing, a validation-guided mechanism
that selectively updates and uploads high-utility layers, reducing drift and commu-
nication overhead; (2) Server-Side Probing, a privacy-preserving method that uses
zero-input embeddings to construct dynamic round-wise similarity graphs; and (3)
Neighbor-Union Layer Aggregation (NULA), a per-layer aggregation strategy that
leverages updates from similar neighbors while preserving personalization. We
evaluate FL-GAP on FDf-27, a federated benchmark derived from DF40 with 27
deepfake methods spanning face swapping, reenactment, synthesis, and editing.
FDf-27 defines five increasingly challenging scenarios, including cross-domain
and globally unseen methods. Experiments show that FL-GAP consistently out-
performs centralized, general FL, and personalized FL baselines, with particularly
strong gains in unseen-method and OOD settings, while cutting communication
by up to 75%.

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 24224