Facial Hair Area in Face Recognition Across Demographics: Small Size, Big Effect

Haiyu Wu, Sicong Tian, Aman Bhatta, Kagan Öztürk, Karl Ricanek, Kevin W. Bowyer

Abstract: Observed variations in face recognition accuracy across demographics, often viewed as “bias”, have motivated re-search into the causes of such variations. Variations in facial hairstyle are an important potential cause of accu-racy differences for males. In this work, we first explore how face recognition accuracy is affected by the facial hair region - clean-shaven, mustache, chin-area beard, side-to-side beard. Results show that mustache area facial hair has a greater effect on accuracy than either chin-area beard or side-to-side beard. We then employ a synthetic facial hair method to verify the consistency of the observation across five synthetic facial hair colors and three face matchers. Re-sults of these experiments indicate that, the larger the dif-ference in brightness between facial hair region and skin region, the larger impact of the mustache area. To reduce accuracy differences caused by facial hairstyle, quantified by $\Delta d^{\prime}$ , we adjust the training dataset distribution to have increased representation of facial hair, resulting in an over 40% reduction in accuracy difference.