Multimodal federated learning: Concept, methods, applications and future directions

Wei Huang, Dexian Wang, Xiaocao Ouyang, Jihong Wan, Jia Liu, Tianrui Li

Abstract: Highlights•The three different modes in the multimodal federated learning model are summarized.•Multimodal fusion based on the federated learning framework is also specified.•The difficulties and some ideas of multimodal federated learning are discussed.