Imputation-free Learning of Tabular Data with Missing Values using Incremental Feature Partitions in Transformer

Manar Samad, Kazi Fuad Bin Akhter, Ibna Kowsar, Shourav B Rabbani

Keywords: missing values, tabular data, attention, imputation, incremental learning

TL;DR: We propose an imputation-free method for deep learning of tabular data without requiring initialization, imputation, and exclusion of missing value to maintain data quality.

Abstract: Tabular data with varying missing values are imputed using an arbitrary imputation strategy for machine learning, which often compromises the data quality and reliability of data-driven outcomes. This article proposes imputation-free incremental attention learning (IFIAL) to streamline tabular data in a transformer without requiring initialization, imputation, or complete representation of missing values. A pair of attention masks are derived and retrofitted to the transformer, which incrementally learns small partitions of overlapping feature sets to enhance the efficiency and performance of learning representations. The average classification performance rank across 17 diverse tabular data sets shows the superiority of IFIAL over 11 state-of-the-art learning methods with or without missing value imputations. Further experiments substantiate the robustness of IFIAL against the varying types and rates of missing values. The proposed method is one of the first solutions to enable deep attention learning of tabular data without requiring missing-value imputation or learning a complete data representation for classification. The source code for this paper is made available.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 3149