Spatial-temporal Semantic Communications for Point Cloud-based Volumetric Media

Jingxuan Men, Ning Wang, Yi Ma, Peng Qian, Carl C. Udora, Mike Nilsson

Abstract: As a representative application paradigm for supporting volumetric video, point cloud is able to capture three-dimensional spatial information of objects, offering dynamic and immersive experiences to end users. Compared to the conventional bit-to-bit communication principle, the design of semantic communication ingeniously utilizes joint source and channel coding to transmit semantic features instead. By taking into account the common content delivery requirements of low bandwidth consumptions and low latency, while maintaining high resolutions, we present in this paper a novel framework called Spatial-Temporal Semantic Point Cloud Transmission (ST-SPCT). Compared to the existing point cloud compression and semantic communication methods that extract and reconstruct features only in the spatial dimension, the ST-SPCT simultaneously extracts and reconstructs spatial-temporal semantic features more deeply, thus significantly reducing computational time and data volume, while ensuring negligible compromises in peak signal-to-noise ratio (PSNR), chamfer distance (CD) metrics, and frame continuity according to our systematic experiment results.