Understanding the Limitations of B-Spline KANs: Convergence Dynamics and Computational Efficiency
TL;DR: KANs have been proposed as a potential alternative to MLPs but when benchmarked on real-world tasks, they consistently under-perform compared to MLPs
Abstract: Kolmogorov-Arnold Networks (KANs) have recently emerged as a potential alternative to multi-layer perceptrons (MLPs), leveraging the Kolmogorov Representation Theorem to introduce learnable activation functions on each edge rather than fixed activations at the nodes. While KANs have demonstrated promise in small-scale problems by achieving similar or better performance with fewer parameters, our empirical investigations reveal significant limitations when scaling to real-world tasks. Specifically, KANs suffer from increased computational costs and reduced performance, rendering them unsuitable for deep learning applications. Our study explores these limitations through extensive testing across diverse tasks, including computer vision and scientific machine learning, and provides a detailed comparison with MLPs.
Style Files: I have used the style files.
Submission Number: 31
Loading