Turning the Tables: Biased, Imbalanced, Dynamic Tabular Datasets for ML EvaluationDownload PDF

06 Jun 2022, 16:31 (modified: 14 Oct 2022, 18:42)NeurIPS 2022 Datasets and Benchmarks Readers: Everyone
Keywords: tabular data, bias, dynamic environments, fairness, fraud detection
TL;DR: A suite of realistic tabular datasets with different biased patterns.
Abstract: Evaluating new techniques on realistic datasets plays a crucial role in the development of ML research and its broader adoption by practitioners. In recent years, there has been a significant increase of publicly available unstructured data resources for computer vision and NLP tasks. However, tabular data — which is prevalent in many high-stakes domains — has been lagging behind. To bridge this gap, we present Bank Account Fraud (BAF), the first publicly available 1 privacy-preserving, large-scale, realistic suite of tabular datasets. The suite was generated by applying state-of-the-art tabular data generation techniques on an anonymized,real-world bank account opening fraud detection dataset. This setting carries a set of challenges that are commonplace in real-world applications, including temporal dynamics and significant class imbalance. Additionally, to allow practitioners to stress test both performance and fairness of ML methods, each dataset variant of BAF contains specific types of data bias. With this resource, we aim to provide the research community with a more realistic, complete, and robust test bed to evaluate novel and existing methods.
Supplementary Material: zip
URL: https://github.com/feedzai/bank-account-fraud
Dataset Url: https://github.com/feedzai/bank-account-fraud
License: Creative Commons CC BY-NC-ND 4.0
Author Statement: Yes
Contribution Process Agreement: Yes
In Person Attendance: Yes
50 Replies