FEPC: Fairness Estimation Using Prototypes and Critics for Tabular Data

Amit Giloni, Edita Grolman, Yuval Elovici, Asaf Shabtai

Abstract: A machine learning (ML) fairness estimator, which is used to assess an ML model’s fairness, should satisfy several conditions when used in real-life settings. Specifically, it should: i) support a comprehensive fairness evaluation that explores all ethical aspects; ii) be flexible and support different ML model settings; iii) enable comparison between different evaluations and ML models; and iv) provide reasoning and explanations for the fairness assessments produced. Existing methods do not sufficiently satisfy all of the above conditions. In this paper, we present FEPC (Fairness Estimation using Prototypes and Critics for tabular data), a novel method for fairness assessment that provides explanations and reasoning for its assessments by using an adversarial attack and customized fairness measurement. Given an ML model and data records, FEPC performs a comprehensive fairness evaluation and produces a fairness assessment for each examined feature. FEPC was evaluated using two benchmark datasets (ProPublica COMPAS and Statlog datasets) and a synthetic dataset containing two features, one of which is biased and one of which is fair, and compared to existing fairness assessment methods. The evaluation demonstrates that FEPC satisfies all of the conditions, making it suitable for real-life settings, and outperforms existing methods.