Reproducibility Study of "Improvement-Focused Causal Recourse (ICR)"
Abstract: This paper presents a reproducibility study of the "Improvement-Focused Causal Recourse (ICR)" model, a novel approach in the field of algorithmic recourse and fairness. The original work by König et al. (2023) introduces ICR as a method to ensure that interventions in predictive models not only achieve the desired outcome (acceptance) but also lead to genuine improvement in real-world situations. Our study aims to validate and replicate the key claims of the original paper by conducting experiments across four datasets, including fully synthetic and semi-synthetic data. We specifically focus on four main claims: (1) ICR's effectiveness in scenarios where gaming is lucrative, (2) ICR’s ability to achieve acceptance rates comparable to traditional methods like counterfactual explanation (CE) and causal recourse (CR), and (3) ICR's robustness to model re-fitting, and (4) cost of interventions in all methods. Our findings largely corroborate the original claims, with ICR demonstrating superior performance in guiding towards actual improvements and maintaining stable acceptance rates despite model re-fitting, a notable advantage over CE and CR methods. While we observe minor numerical discrepancies in results, the overall trends align with the original study, reinforcing the efficacy of ICR in enhancing both the explainability and equity of automated decision systems. This reproducibility study not only confirms the original findings but also highlights the importance of robust and practical approaches in algorithmic recourse for real-world applications.
Submission Length: Regular submission (no more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=lDk4kCGZLQ
Changes Since Last Submission: We have made substantial enhancements to our work following the valuable feedback received. In the revised methodology section, we now provide concrete examples that elucidate the theoretical underpinnings of Causal Recourse. Furthermore, we have explicitly addressed the limitations of our study, particularly focusing on our approach's constrained capability to evaluate the correctness of the specified SCMs. Additionally, in response to suggestions for improving the clarity of our presentation, we have increased both the size and quality of the visual aids used throughout our work.
Assigned Action Editor: ~Fredrik_Daniel_Johansson1
Submission Number: 2235
Loading