No imputation without representation

TMLR Paper335 Authors

Abstract: By filling in missing values in datasets, imputation allows these datasets to be used with algorithms that cannot handle missing values by themselves. However, missing values may in principle contribute useful information that is lost through imputation. The missing-indicator approach can be used in combination with imputation to instead represent this information as a part of the dataset. There are several theoretical considerations why missing-indicators may or may not be beneficial, but there has not been any large-scale practical experiment on real-life datasets to test this question for machine learning predictions. We perform this experiment for three imputation strategies and a range of different classification algorithms, on the basis of twenty real-life datasets. We find that on these datasets, missing-indicators generally increase classification performance. In addition, we find no evidence for most algorithms that nearest neighbour and iterative imputation lead to better performance than simple mean/mode imputation. Therefore, we recommend the use of missing-indicators with mean/mode imputation as a safe default, with the caveat that for decision trees, pruning is necessary to prevent overfitting. In a follow-up experiment, we determine attribute-specific missingness thresholds for each classifier above which missing-indicators are more likely than not to increase classification performance, and observe that these thresholds are much lower for categorical than for numerical attributes. Finally, we argue that mean imputation of numerical attributes may preserve some of the information from missing values, and we show that in the absence of missing-indicators, it can similarly be useful to apply mean imputation to one-hot encoded categorical attributes instead of mode imputation.

Submission Length: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=QmOESArhpp&noteId=v4QtDTUjSc

Changes Since Last Submission:

See comments to the reviewers. In particular:

* Appendix with AUROC tables
* New Subsection 3.1 with the three questions that the experiments try to answer, renamed subsections of Section 4 (Results) to match these questions
* Added references to classifier-specific missing value solutions to Subsection 3.4
* Added short discussion of "missingness incorporated in attributes" (MIA) to Subsection 2.2 (Previous experiments)
* Added paragraph to introduction to discuss the possibility that missingness-information may be recoverable from imputated values ("imputation manifolds")

Assigned Action Editor: ~antonio_vergari2

Submission Number: 335