Automated Counterfactual Generation in Financial Model Risk Management

Jingwei Gan, Shinan Zhang, Chi Zhang, Andy Li

Abstract: The increasing adoption of machine learning and AI models in finance and banking poses challenges for model testing and model risk management (MRM). Conventional testing has limitations in evaluating more complex models while the latest testing methodologies have not yet been widely productionized. We propose a model-agnostic testing algorithm based on counterfactual generation for machine learning models in finance. Our method provides actionable counterfactual samples that can guide future model analysis and improvement. To make our algorithm modeler-friendly, we design a cloud-native implementation that is easy to integrate with any new and existing models, and ready for production-level automation, scalability, and parallel processing.