Perception-Driven Point Cloud Quality Assessment Through Projections and Deep Structure Similarity

Arthur H. S. Carvalho, Pedro Garcia Freitas, Mateus Gonçalves, Johann Homonnai, Mylène C. Q. Farias

Abstract: Point Clouds (PCs) have gained considerable interest as a potential format for representing tridimensional (3D) data in various applications, including augmented reality and autonomous vehicles. In the realm of multimedia, where applications and technologies such as 3DTV revolve around human interaction, it is crucial to assess how humans perceive the visual quality of PCs. To address this need, research on Point Cloud Quality Assessment (PCQA) metrics, incorporating aspects of the human visual perception, has gained prominence. This research aims to facilitate PC quality enhancement, optimize PC registration pipelines, and improve the performance of PC codecs. However, assessing the quality of PCs poses inherent challenges due to its irregularity and sparsity, necessitating the identification of mathematical relationships within PCs for accurate quality predictions. In this work, we adopt a projection-based approach, representing a PC as a structured set of bidimensional (2D) projections-regular images derived from the 3D structure of the PC. We leverage the power of Neural Networks (NNs) and Machine Learning (ML) to create a perceptual-driven, full-reference metric for PCQA. We assess these images using established Image Quality Assessment (IQA) methods, widely acknowledged in the state-of-the-art (SOTA) for traditional 2D (non-immersive) visual content. The generated scores are then deposited into a vector, serving as input for a regressor model to predict the final quality score. Our results demonstrate the competitiveness of our model compared to SOTA metrics.