Sparse Interaction Additive Networks via Feature Interaction Detection and Sparse Selection

James Enouen, Yan Liu

Keywords: interpretability, additive models

TL;DR: Extends GAM models to DNNs with feature interactions of size three and higher

Abstract:

There is currently a large gap in performance between the statistically rigorous methods like linear regression or additive splines and the powerful deep methods using neural networks. Previous works attempting to close this gap have failed to fully consider the exponentially growing number of feature combinations which deep networks consider automatically during training. In this work, we develop a tractable selection algorithm to efficiently identify the necessary feature combinations by leveraging techniques in feature interaction detection.
Our proposed Sparse Interaction Additive Networks (SIAN) construct a bridge from these simple and interpretable models to a fully connected neural network. SIAN achieves competitive performance against state-of-the-art methods across multiple large-scale tabular datasets and consistently finds an optimal tradeoff between the modeling capacity of neural networks and the generalizability of simpler methods.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2209.09326/code)