Adaptive Calibration for Fairer Facial Recognition

Ryan Brown, Chris Russell

Keywords: facial recognition, fairness, calibration

TL;DR: We introduce Adaptive Calibration, a post-hoc method that has sota fairness-accuracy trade-offs for facial recognition.

Abstract:

We introduce a novel calibration strategy for facial recognition, Adaptive Calibration, which maps cosine similarity between normalized embeddings to well-calibrated probabilities. By incorporating local embedding context into the calibration process, Adaptive Calibration is able to correct for the fact identical distances correspond to different match probabilities in different embedding regions. This yields improved calibration that adapts to local embedding distributions without requiring demographic metadata.

Experiments with standard benchmarks for face verification across a variety of pretrained models demonstrate that our approach consistently dominates existing methods both on accuracy (AUROC) and fairness metrics. Our method provides a practical solution for more equitable facial recognition systems, without requiring demographic group annotations, and while improving overall performance. Unlike existing approaches that often rely on discrete clustering with additional hyperparameters or cause abrupt calibration shifts at cluster boundaries, our method provides continuous, region-specific calibration that avoids both the algorithmic limitations and the issue of ``leveling down'' whereby fairness is achieved by degrading performance for already disadvantaged groups.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 11092