Unsupervised Deep Comparison For Face Anomaly Appraisal

Abdullah Hayajneh, Erchin Serpedin, Mitchell A. Stotland

Keywords: Anomaly Detection, Generative Adversarial Networks, Deep Feature Comparison

Abstract: This paper proposes an innovative anomaly appraisal framework that combines machine learning and signal processing for the consistent detection, localization, and evaluation of anomalies in human faces. The primary objective of this framework is to create a universal and objective metric for evaluating the degree of facial anomaly and reconstructive surgical outcomes. This metric should be well aligned with the human assessments. To accomplish this, the framework leverages the StyleGAN2 facial generator to normalize human faces that may exhibit diverse deformities. The proposed method utilizes a pre-trained Convolutional Neural Network (CNN) to extract and compare deep anomalous features between the original image and its normalized counterpart, in an unweighted manner. The resulting anomaly maps are merged into a heatmap, effectively highlighting the abnormal facial regions. This heatmap is then employed to generate a machine score, quantifying the degree of anomaly in the face. In order to assess the effectiveness of the proposed method, a subjective comparison of the generated anomaly maps is conducted using metrics such as Learned Perceptual Image Patch Similarity (LPIPS), Structural Similarity Index (SSIM), Pixelwise Subtraction (PS), alongside the newly introduced method. This comparative analysis demonstrates the model's robust performance, exhibiting the highest linear (0.92 Pearson’s r score) and monotonicity correlation (0.85 Spearman's $\rho$ score) between human and generated scores. These results corroborate the model's efficacy and its close alignment with natural human judgment across different levels of facial anomalies.

Track: 5. Biomedical generative AI

Registration Id: 5KNRM3ZNK65

Submission Number: 5