Robust Deep Neural Networks for Heterogeneous Tabular Data Download PDF

Published: 28 Jan 2022, Last Modified: 13 Apr 2023ICLR 2022 SubmittedReaders: Everyone
Keywords: Deep Learning, Deep Neural Networks, Tabular Data, Gradient Boosted Decision Trees
Abstract: Although deep neural networks (DNNs) constitute the state-of-the-art in many tasks based on image, audio, or text data, their performance on heterogeneous, tabular data is typically inferior to that of decision tree ensembles. To bridge the gap between the difficulty of DNNs to handle tabular data and leverage the flexibility of deep learning under input heterogeneity, we propose DeepTLF, a framework for deep tabular learning. The core idea of our method is to transform the heterogeneous input data into homogeneous data to boost the performance of DNNs considerably. For the transformation step, we develop a novel knowledge distillations approach, TreeDrivenEncoder, which exploits the structure of decision trees trained on the available heterogeneous data to map the original input vectors onto homogeneous vectors that a DNN can use to improve the predictive performance. Through extensive and challenging experiments on various real-world datasets, we demonstrate that the DeepTLF pipeline leads to higher predictive performance. On average, our framework shows 19.6\% performance improvement in comparison to DNNs. The DeepTLF code is publicly available.
One-sentence Summary: Making deep neural networks robust to tabular data quality issues by utilizing decision tree ensembles for data encoding.
Supplementary Material: zip
12 Replies

Loading