Cross Spline Net: A Simpler, More Interpretable and Unified Machine Learning Framework

Linwei Hu, Ye Jin Choi, Zheng Yuan, Sumegha Premchandar, Freddy Lecue

Keywords: Cross net, spline transformation, model interpretability, functional-anova

TL;DR: A flexible and convenient modeling framework for performant, robust and more interpretable models, along with model interpretation tools

Abstract: We propose a new machine learning framework called cross spline net (XSN), along with a set of interpretability methods to understand the model. The framework is built on a combination of spline transformation and cross-network (Wang et al. 2017, 2021). Compared to the traditional black-box machine learning models like XGBoost and fully connected neural network (FCNN), the XSN framework has a few key advantages. First, it is simpler and less overfitted while being as performant or better in some cases. Second, it is flexible and unifies a variety of machine learning models under the same framework. With different choices of the spline layer, we can reproduce or approximate a set of non-neural network models (tree, MARS, SVM, etc.) under the same, unified neural network framework. By using scalable and powerful optimization algorithms available in neural network libraries, XSN avoids some pitfalls (such as being ad-hoc, greedy or non-scalable) in the original optimization methods used in these non-neural network models. Finally, we equip XSN with a set of interpretability tools that help users understand the model composition and feature effects, which is crucial to gain insights and confidence in the deployed model. We will use a special type of XSN, TreeNet, to illustrate our point. We believe XSN will provide a flexible and convenient framework for practitioners to build performant, robust and more interpretable models.

Primary Area: interpretability and explainable AI

Submission Number: 11865