Less is More in Kolmogorov-Arnold Networks: The Shared-Atom Architecture

Keywords: Parameter-efficient deep learning, Kolmogorov-Arnold Networks, Dictionary learning, Low-rank factorization, Interpretable neural networks, Scientific machine learning, Symbolic regression

TL;DR: SA-KAN is a more efficient version of KANs Rather than learning a separate function for each edge, it learns a small shared set of reusable functions (“atoms”) and mixes them sparsely where needed.

Abstract:

Standard MLPs use fixed nonlinearities, which can limit interpretability despite strong universal approximation properties. Kolmogorov-Arnold Networks (KANs) address this issue by assigning learnable univariate functions to edges, but this induces a prohibitively large parameter tensor and corresponding scaling difficulties. We develop Shared-Atom KAN (SA-KAN) by enforcing a low-rank functional factorization in place of independent edge functions. SA-KAN learns a dictionary of $K$ reusable atoms that are combined through sparse edge-wise coefficients, effectively separating the choice of basis from network architecture and compressing the layer into a symbolic vocabulary.
Empirically, this formulation reduces parameter counts by 25--30\% relative to Vanilla KAN while simultaneously improving performance on structured tasks.

Submission Number: 126