Deep Neural Forests: An Architecture for Tabular Data

Ami Abutbul, Gal Elidan, Liran Katzir, Ran El-Yaniv

TL;DR: An architecture for tabular data, which emulates branches of decision trees and uses dense residual connectivity

Abstract:

Deep neural models, such as convolutional and recurrent networks, achieve phenomenal results over spatial data such as images and text.
However, when considering tabular data, gradient boosting of decision trees (GBDT) remains the method of choice.
Aiming to bridge this gap, we propose \emph{deep neural forests} (DNF)
-- a novel architecture that combines elements from decision trees as well as dense residual connections.
We present the results of extensive empirical study in which we examine the performance of GBDTs, DNFs and (deep) fully-connected networks.
These results indicate that DNFs achieve comparable results to GBDTs on tabular data, and open the door to end-to-end neural modeling of multi-modal data. To this end, we present a successful application of DNFs as part of a hybrid architecture for a multi-modal driving scene understanding classification task.

Keywords: neural architectures, tabular data, multi-modal data, decision trees, gradient boosting