class balanced adaptive pseudo labeling for federated semi-supervised learning

Ming Li, Yan Wang, Qingli Li

Abstract:

This paper focuses on federated semi-supervised learning (FSSL), assuming that few clients have fully labeled
data (labeled clients) and the training datasets in other
clients are fully unlabeled (unlabeled clients). Existing
methods attempt to deal with the challenges caused by not
independent and identically distributed data (Non-IID) setting. Though methods such as sub-consensus models have
been proposed, they usually adopt standard pseudo labeling or consistency regularization on unlabeled clients which
can be easily influenced by imbalanced class distribution.
Thus, problems in FSSL are still yet to be solved. To seek
for a fundamental solution to this problem, we present Class
Balanced Adaptive Pseudo Labeling (CBAFed), to study
FSSL from the perspective of pseudo labeling. In CBAFed,
the first key element is a fixed pseudo labeling strategy to
handle the catastrophic forgetting problem, where we keep
a fixed set by letting pass information of unlabeled data
at the beginning of the unlabeled client training in each
communication round. The second key element is that we
design class balanced adaptive thresholds via considering the empirical distribution of all training data in local clients, to encourage a balanced training process. To make
the model reach a better optimum, we further propose a
residual weight connection in local supervised training and
global model aggregation. Extensive experiments on five
datasets demonstrate the superiority of CBAFed. Code will
be available at https://github.com/minglllli/
CBAFed.