Debugging the Black-Box COMPAS Risk Assessment Instrument to Diagnose and Remediate Bias

Patrick Hall, Navdeep Gill

Abstract: The black-box Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) criminal risk assessment instrument (RAI) is analyzed for confounding racial bias and a novel procedure is proposed for remediating bias from individual criminal risk predictions. A repeatable global versus local analysis motif is introduced in which global and local model behavior are compared to debug and diagnose unwanted bias in a black-box prediction system using tools such as surrogate models, gradient boosting machine feature importance, leave-one-covariate-out (LOCO) feature importance, partial dependence plots, and individual conditional expectation (ICE) plots. LOCO-derived feature importance is also used to remove prediction contributions from bias-inducing input features. The proposed global versus local approach and remediation strategy can be applied to many blackbox and machine learning (ML) decision-making systems.

TL;DR: The black-box COMPAS criminal risk prediction instrument is debugged for unwanted racial bias and racial bias is numerically remediated from it's predictions.

Keywords: Machine Learning, COMPAS, Decision Tree, GBM, Surrogate Model, Partial Dependence, ICE, Variable Importance, LOCO, Interpretability, Inference