Contrastive Learning Enables Low-Bandwidth Semantic Communication

Eleonora Grassucci, Giordano Cicchetti, Danilo Comminiello

Keywords: Multimodal Learning, Semantic Communication, Multimodal Alignment, Contrastive Learning

TL;DR: We propose a novel semantic communication framework that shapes a semantically aligned latent space by means of contrastive learning

Abstract: The rapid growth of multimodal data streaming requiring more and more bandwidth is posing new challenges for communication systems. Concurrently, by transmitting only the semantic information and not the whole original bitstream, the novel semantic communication paradigm promises to reduce the bandwidth requirements. However, for multimodal data transmission, conventional semantic communication frameworks also require conveying a considerable amount of information for each modality, resulting in a high transmission load. In this paper, we propose to model the semantic latent space with a novel contrastive learning loss, so as to extract the centroid representing the semantic content of the respective cluster and transmit over the channel just one single compressed representation, regardless of the number of modalities. We show how the proposed framework allows a considerable reduction of the bandwidth while preserving multimodal reconstruction results with respect to conventional approaches.

Submission Number: 3