Learning Expressive Representations with Kolmogorov-Arnold Network Autoencoders

TMLR Paper6241 Authors

Abstract: Processing high-dimensional data presents significant challenges, including the curse of dimensionality and the limitations of models with static, non-adaptive components. While autoencoders are a cornerstone of unsupervised representation learning, their performance is often constrained by traditional activation functions. Inspired by the Kolmogorov-Arnold representation theorem, this paper introduces Kolmogorov-Arnold Network Autoencoders (KAN-AEs), a novel framework that replaces static activations with learnable, spline-based functions. We further propose a Convolutional KAN-AE (CKAN-AE) variant that incorporates spatial inductive biases for image data. Through comprehensive experiments on benchmark datasets, we demonstrate that KAN-based autoencoders consistently achieve superior reconstruction fidelity and learn more discriminative latent representations, as evidenced by improved linear probing accuracy. Notably, CKAN-AE excels on complex natural images. While the enhanced expressiveness introduces a computational trade-off, our work establishes KAN-AEs as a powerful tool for scenarios where representation quality is paramount, paving the way for more adaptive and efficient deep learning models.

Submission Length: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Yingnian_Wu1

Submission Number: 6241