From 2D Images to 3D Model: Segmenting and Reconstructing Acne for Robust Skin Lesion Analysis

Chengrui Zhang, Zhaorui Tan, Xi Yang, Liqiong Yuan, Zhanbao Su, Amir Hussain, Yifei Zhang, Bin Dong, Jie Sun, Kaizhu Huang

Abstract: Acne is a prevalent skin disorder causing significant physical and psychological distress. Effective treatment relies on accurate diagnosis and monitoring, but conventional analysis using 2D facial images is limited by issues like self-occlusion and lacks quantitative depth, failing to capture the true extent of the condition. To address this problem, we propose a novel system that integrates deep learning-based acne segmentation with 3D facial reconstruction from a few non-simultaneously captured 2D images. Our methodology first employs a state-of-the-art segmentation model to accurately identify acne lesions from multi-view facial photographs. It then uses a novel framework to reconstruct a detailed 3D facial model, even from unconstrained images, and maps the segmented acne onto this model. Experimental results demonstrate that our system effectively generates accurate 3D face models with integrated acne distributions. The TransUNet model achieved superior segmentation performance with an F1-Score of 0.7765, and our 3D reconstruction method surpassed existing techniques with an average PSNR of 24.36 and SSIM of 0.92. This approach provides clinicians with a comprehensive tool for improved diagnosis, personalized treatment planning, and objective monitoring of skin lesions.

External IDs:dblp:journals/cogcom/ZhangTYYSHZDSH26