Multimodal and Multiuser Semantic Communications for Channel-Level Information Fusion

Xuewen Luo, Ruobin Gao, Hsiao-Hwa Chen, Shuyi Chen, Qing Guo, Ponnuthurai Nagaratnam Suganthan

Abstract: Recently, semantic communication emerged as one of the enabling technologies for intelligent communications and multimodal information fusion plays an important role therein, where various sensors collect environment information to generate complementary multimodal data. In this article, we propose a new multimodal information fusion scheme for multi-user semantic communications, namely channel-level information fusion, taking into account the nature of signal transmission in wireless channels. In the proposed scheme, wireless channel acts as a medium to fuse multi-modal data and the received signal is viewed as the fused information, such that a receiver can retrieve semantic information without a need to perform multiuser signal detection. In addition, semantic precoding is used to mitigate the randomness effect of wireless channels in fusion. A case study is conducted to validate the effectiveness of the proposed fusion scheme, followed by a discussion on the open issues and challenges to be tackled.