Open Peer Review. Open Publishing. Open Access. Open Discussion. Open Directory. Open Recommendations. Open API. Open Source.
Debugging the Black-Box COMPAS Risk Assessment Instrument to Diagnose and Remediate Bias
Patrick Hall, Navdeep Gill
Jun 14, 2017 (modified: Jun 19, 2017)ICML 2017 WHI Submissionreaders: everyone
Abstract:The black-box Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) criminal risk assessment instrument (RAI) is analyzed for confounding racial bias and a novel procedure is proposed for remediating bias from individual criminal risk predictions. A repeatable global versus local analysis motif is introduced in which global and local model behavior are compared to debug and diagnose unwanted bias in a black-box prediction system using tools such as surrogate models, gradient boosting machine feature importance, leave-one-covariate-out (LOCO) feature importance, partial dependence plots, and individual conditional expectation (ICE) plots. LOCO-derived feature importance is also used to remove prediction contributions from bias-inducing input features. The proposed global versus local approach and remediation strategy can be applied to many blackbox and machine learning (ML) decision-making systems.
TL;DR:The black-box COMPAS criminal risk prediction instrument is debugged for unwanted racial bias and racial bias is numerically remediated from it's predictions.