Relating Piecewise Linear Kolmogorov Arnold Networks to ReLU Networks

Nandi Schoots, Mattia Jacopo Villani, Niels uit de Bos

TL;DR: We provide completely explicit constructions to convert a piecewise linear Kolmogorov Arnold Network into a ReLU network and vice versa.

Abstract:

Kolmogorov-Arnold Networks are a new family of neural network architectures which holds promise for overcoming the curse of dimensionality and has interpretability benefits (Liu et al., 2024). In this paper, we explore the connection between Kolmogorov Arnold Networks (KANs) with piecewise linear (univariate real) functions and ReLU networks. We provide completely explicit constructions to convert a piecewise linear KAN into a ReLU network and vice versa.

Submission Number: 89