A Graph Meta-Network for Learning on Kolmogorov–Arnold Networks

ICLR 2026 Conference Submission19899 Authors

Keywords: Weight Space, Kolmogorov Arnold Networks, Graph Neural Networks, Symmetries, Equivariance

TL;DR: We introduce WS-KAN the first weight-space model for KANs, respecting their permutation symmetries via a graph representation. It achieves strong expressiveness and consistently outperforms baselines.

Abstract:

Weight-space models learn directly from the parameters of neural networks, enabling tasks such as predicting their accuracy on new datasets. Naive methods -- like applying MLPs to flattened parameters -- perform poorly, making the design of better weight-space architectures a central challenge. While prior work leveraged permutation symmetries in standard networks to guide such designs, no analogous analysis or tailored architecture yet exists for Kolmogorov–Arnold Networks (KANs). In this work, we show that KANs share the same permutation symmetries as MLPs, and propose the KAN-graph, a graph representation of their computation.
Building on this, we develop WS-KAN, the first weight-space architecture that learns on KANs, which naturally accounts for their symmetry. We analyze WS-KAN’s expressive power, showing it can replicate an input KAN’s forward pass - a standard approach for assessing expressiveness in weight-space architectures. We construct a comprehensive ``zoo'' of trained KANs spanning diverse tasks, which we use as benchmarks to empirically evaluate WS-KAN. Across all tasks, WS-KAN consistently outperforms structure-agnostic baselines, often by a substantial margin.

Primary Area: learning on graphs and other geometries & topologies

Submission Number: 19899