Federated Multimodal Learning for IoT Applications: A Contrastive Learning Approach

Huy Q. Le, Yu Qiao, Loc X. Nguyen, Luyao Zou, Choong Seon Hong

Abstract: The increasing use of IoT devices has led to the generation of vast amounts of data from various modalities, making them ideal candidates for federated learning (FL). FL is a machine learning approach that allows models to be trained on decentralized data sources without compromising data privacy and security, making it a suitable technique for IoT applications. However, existing FL methods mainly focus on unimodal data, which limits their applicability in real-world IoT applications where devices consist of data from multiple sources. To address this limitation, we propose a Federated Multimodal Learning approach for IoT applications with a dual contrastive regularization (DC-MMFed). Our proposed method enables clients to learn a joint multimodal representation from multimodal data while preserving data privacy. By using contrastive learning, our method allows for learning discriminative features across different modalities. We evaluate our approach on a human activity recognition dataset and demonstrate its superior performance on different downstream tasks compared to baseline FL methods. Our work contributes to privacy-preserving multimodal machine learning in IoT applications, advancing network management without compromising data security. By leveraging DC-MMFed, devices can perform more accurate and robust machine learning tasks without data centralization or sharing, thus maintaining data privacy and security.

External IDs:dblp:conf/apnoms/LeQNZH23