Numerical Data Imputation for Multimodal Data Sets: A Probabilistic Nearest-Neighbor Kernel Density Approach

Florian Lalande, Kenji Doya

Abstract:

Numerical data imputation algorithms replace missing values by estimates to leverage incomplete data sets. Current imputation methods seek to minimize the error between the unobserved ground truth and the imputed values. But this strategy can create artifacts leading to poor imputation in the presence of multimodal or complex distributions. To tackle this problem, we introduce the $k$NN$\times$KDE algorithm: a data imputation method combining nearest neighbor estimation ($k$NN) and density estimation with Gaussian kernels (KDE).
We compare our method with previous data imputation methods using artificial and real-world data with different data missing scenarios and various data missing rates, and show that our method can cope with complex original data structure, yields lower data imputation errors, and provides probabilistic estimates with higher likelihood than current methods. We release the code in open-source for the community.

Certifications: Reproducibility Certification

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

Final version of the manuscript.

We tried to address the concerns of reviewer [RUkZ]:
- Clearer explanation of the difference between point estimate and our method that returns a distribution.
- Explain that what we call "MICE" in our manuscript uses the linear regression (standard version), and we made it clearer that MissForest is also a "MICE"-type algorithm, with a different regressor.

In addition:
- We provide a link to our GitHub repository. All the code, data sets, and results are open sourced at https://github.com/DeltaFloflo/knnxkde.
- We created a friendly Jupyter Notebook to help researchers/practitioners get started with the $k$NN$\times$KDE.
- We de-anonymize the paper

Code: https://github.com/DeltaFloflo/knnxkde

Assigned Action Editor: ~Pierre_Alquier1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 1046