Debugging the Black-Box COMPAS Risk Assessment Instrument to Diagnose and Remediate Bias

Patrick Hall, Navdeep Gill

Jun 14, 2017 (modified: Jun 19, 2017) ICML 2017 WHI Submission readers: everyone
  • Abstract: The black-box Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) criminal risk assessment instrument (RAI) is analyzed for confounding racial bias and a novel procedure is proposed for remediating bias from individual criminal risk predictions. A repeatable global versus local analysis motif is introduced in which global and local model behavior are compared to debug and diagnose unwanted bias in a black-box prediction system using tools such as surrogate models, gradient boosting machine feature importance, leave-one-covariate-out (LOCO) feature importance, partial dependence plots, and individual conditional expectation (ICE) plots. LOCO-derived feature importance is also used to remove prediction contributions from bias-inducing input features. The proposed global versus local approach and remediation strategy can be applied to many blackbox and machine learning (ML) decision-making systems.
  • TL;DR: The black-box COMPAS criminal risk prediction instrument is debugged for unwanted racial bias and racial bias is numerically remediated from it's predictions.
  • Keywords: Machine Learning, COMPAS, Decision Tree, GBM, Surrogate Model, Partial Dependence, ICE, Variable Importance, LOCO, Interpretability, Inference