KAAN: Kolmogorov-Arnold Activation Network --- a Flexible Activation Enhanced KAN
Keywords: Kolmogorov-Arnold representation Theorem, Kolmogorov-Arnold Network, Multi-Layer Perceptrons
Abstract: Kolmogorov-Arnold Networks (KANs) have led to a significant breakthrough in the foundational structures of machine learning by applying the Kolmogorov-Arnold representation theorem. Through this approach, the target conditional distribution is expressed as the summation of multiple continuous univariate B-spline functions. The unique and complex computational structure of B-splines makes it hard to understand directly since the properties of each grid are not determined by its own parameters but are also influenced by the parameters of adjacent grids. Besides, it is challenging to trim and splice at components level under B-spline. To address this issue, we analyze the structural configurations of Multi-Layer Perceptrons (MLPs) and KANs, finding that MLP can be represented in a form conforming to Kolmogorov-Arnold representation Theorem (KAT). Therefore, we propose MLP style KAN framework Kolmogorov-Arnold Activation Network (KAAN), which is more straightforward, flexible and transferable. To verify the flexibility and transferability of our approach, we extend it to Convolutional Neural Network (CNN). Also, we demonstrate that parameter sharing is beneficial not only for efficiency but also for effectiveness. KAAN shows better representation capacity than MLP on several benchmarks. Furthermore, our experiment results lead us to conclude that this method is feasible for integrating modern network approaches such as CNNs.
Supplementary Material: zip
Primary Area: other topics in machine learning (i.e., none of the above)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 6995
Loading