Robustness Disparities in Commercial Face Detection

Samuel Dooley, Tom Goldstein, John P Dickerson

Abstract: Facial detection and analysis systems have been deployed by large companies and critiqued by scholars and activists for the past decade. Critiques that focus on system performance analyze disparity of the system's output, i.e., how frequently is a face detected for different Fitzpatrick skin types or perceived genders. However, we focus on the robustness of these system outputs under noisy natural perturbations. We present the first of its kind detailed benchmark of the robustness of two such systems: Amazon Rekognition and Microsoft Azure. We use both standard and recently released academic facial datasets to quantitatively analyze trends in robustness for each. Qualitatively across all the datasets and systems, we find that photos of individuals who are \emph{older}, \emph{masculine presenting}, of \emph{darker skin type}, or have \emph{dim lighting} are more susceptible to errors than their counterparts in other identities.

URL: https://github.com/dooleys/Robustness-Disparities-in-Commercial-Face-Detection

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2108.12508/code)