Detecting Soft-Biometric Privacy Enhancement

Peter Rot, Peter Peer, Vitomir Struc

Abstract: With the proliferationSoft-biometric privacy of facial analytics and automatic recognition technology that can automatically extract a broad range of attributes from facial images, so-called soft-biometric privacy-enhancing techniquesPrivacy-enhancing technique Soft-biometric privacy Biometrics have seen increased interest from the computer vision community recently. Such techniques aim to suppress information on certain soft-biometricSoft-biometric attributes (e.g., age, gender, ethnicity) in facial images and make unsolicited processing of the facial data infeasible. However, because the level of privacy protection ensured by these methods depends to a significant extent on the fact that privacy-enhanced images are processed in the same way as non-tampered images (and not treated differently), it is critical to understand whether privacy-enhancing manipulations can be detected automatically. To explore this issue, we design a novel approach for the detection of privacy-enhanced images in this chapter and study its performance with facial images processed by three recent privacy models. The proposed detection approach is based on a dedicated attribute recovery procedure that first tries to restore suppressed soft-biometricSoft-biometric information and based on the result of the restoration procedure then infers whether a given probe image is privacy enhanced or not. It exploits the fact that a selected attribute classifier generates different attribute predictions when applied to the privacy-enhanced and attribute-recovered facial images. This prediction mismatch (PREM)PREdiction Mismatch (PREM) is, therefore, used as a measure of privacy enhancementPrivacy enhancement. In extensive experiments with three popular face datasets we show that the proposed PREMPREdiction Mismatch (PREM) model is able to accurately detect privacy enhancementPrivacy enhancement in facial images despite the fact that the technique requires no supervision, i.e., no examples of privacy-enhanced images are needed for training.