Re-calibrated Wasserstein GAN for large-scale imputation with informative missing

Mark He, Yiming Li, Katharina Schultebraucks, Ian Kronish, Jianhua Hu, Ying Wei

Keywords: deep learning, data imputation, missing data, neural networks, Wasserstein GAN, quantile regression

TL;DR: We develop a novel method for imputing missing data in large scale health records using a Wasserstein GAN whose loss function is reweighted by missingness probability estimates

Abstract: Missing data are pervasive in electronic health records (EHR) and oftentimes the missingness is informative (i.e. Missing Not At Random). Presently available imputation methods typically do not account for this informative missingness or are computationally infeasible to handle the scale of EHR data. We develop a deep learning imputation method based on \textit{recalibrating} a Wasserstein Generative Adversarial Network (WGAN) to account for informative missingness in high-dimensional quantitative medical data. We propose a new quantile re-weighting technique to ensure distributional equivariance under informative missingness and integrate it with WGAN to enable efficient imputations in large-scale observational data in presence of informative missingness and covariate imbalance. Results from our proposed algorithm show better recovery compared to present methods in both synthetic and real-world data from the Reactions to Acute Hospitalization (REACH) and laboratory test results of COVID-19 patients in the New York Metropolitan area from the INSIGHT dataset.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Machine Learning for Sciences (eg biology, physics, health sciences, social sciences, climate/sustainability )