DOFEN: Deep Oblivious Forest ENsemble

Kuan-Yu Chen, Ping-Han Chiang, Hsin-Rung Chou, Chih-Sheng Chen, Tien-Hao Chang

Keywords: Tabular Data, Structured Data, Deep Neural Network, Architecture Design

TL;DR: DOFEN (Deep Oblivious Forest ENsemble): a novel deep neural network architecture for tabular data, achieving sota performance compared to deep nerual network baselines and comparable performance with tree-based models.

Abstract: Deep Neural Networks (DNNs) have revolutionized artificial intelligence, achieving impressive results on diverse data types, including images, videos, and texts. However, DNNs still lag behind Gradient Boosting Decision Trees (GBDT) on tabular data, a format extensively utilized across various domains. This paper introduces DOFEN, which stands for Deep Oblivious Forest ENsemble. DOFEN is a novel DNN architecture inspired by oblivious decision trees and achieves on-off sparse selection of columns. DOFEN surpasses other DNNs on tabular data, achieving state-of-the-art performance on the well-recognized benchmark: Tabular Benchmark, which includes 73 total datasets spanning a wide array of domains. The code of DOFEN is available at: https://github.com/Sinopac-Digital-Technology-Division/DOFEN

Primary Area: Deep learning architectures

Submission Number: 2708