Personalized Federated Semi-Supervised Learning with Black-Box Models

Siyin Huang, Shao-Yuan Li, Songcan Chen

Abstract:

Federated Semi-Supervised Learning alleviates the
necessity for fully labeled data in Federated Learning. However, it
does not sufficiently prioritize model privacy or the personalized
requirements of clients. To address these concerns, our key idea
is to communicate no longer individual model parameters but their
black-box models, implying to provide other clients with only an
input-output interface of individual models. The communication
mechanism is model-agnostic, thereby facilitating adaption to
heterogeneous models for each client through customization. Consequently, we propose a framework called Personalized Federated
Semi-Supervised Learning with Black-Box Models (B2PFSSL)
to enhance the privacy of communication. To prevent negative
knowledge transfer due to data heterogeneity, we design a twostage strategy that filters at both the model and data levels,
enabling clients to obtain large training datasets by including
more high-quality pseudo-labeled data under conditions of scarce
labeled data. The experimental results indicate that B2PFSSL
achieves competitive performance while reducing the amount
of information exposed during communication. Furthermore,
it can foster productive collaboration among diverse model
architectures in model heterogeneous Federated Learning.