PROMISSING: Pruning Missing Values in Neural Networks

Seyed Mostafa Kia, Nastaran Mohammadian Rad, Daniel van Opstal, Bart van Schie, Wiepke Cahn, Andre Marquand, Josien P.W. Pluim, Hugo G. Schnack

Keywords: Neural Networks, Missing Values, Data Imputation

Abstract: While data are the primary fuel for machine learning models, they often suffer from missing values, especially when collected in real-world scenarios. However, many off-the-shelf machine learning models, including artificial neural network models, are unable to handle these missing values directly. Therefore, extra data preprocessing and curation steps, such as data imputation, are inevitable before learning and prediction processes. In this study, we propose a simple and intuitive yet effective method for pruning missing values (PROMISSING) during learning and inference steps in neural networks. In this method, there is no need to remove or impute the missing values; instead, the missing values are treated as a new source of information (representing what we do not know). Our experiments on simulated data, several classification and regression benchmarks, and a multi-modal clinical dataset show that PROMISSING results in similar classification performance compared to various imputation techniques. In addition, our experiments show models trained using PROMISSING techniques are becoming less decisive in their predictions when facing incomplete samples with many unknowns. This finding hopefully advances machine learning models from being pure predicting machines to more realistic thinkers that can also say "I do not know" when facing incomplete sources of information.

One-sentence Summary: We present a simple yet effective technique for dealing with missing values in neural networks.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2206.01640/code)