Pick‑and‑Unfold: Mode‑Aware Non‑Linear Tucker Autoencoder for Tensorial Deep Learning

Junjing Zheng, Chengliang Song, Weidong Jiang, Xinyu Zhang

Keywords: tensor autoencoder, non-linear tensor decomposition, image compression

TL;DR: We propose a simple yet effective non-linear tensor autoencoder that strikes a balance between superior reconstruction performance and low computational complexity.

Abstract: High-dimensional data, particularly in the form of high-order tensors, presents a major challenge in deep learning. While various deep autoencoders (DAEs) are employed as basic feature extraction modules, most of them depend on flattening operations that exacerbate the curse of dimensionality, leading to excessively large model sizes, high computational overhead, and challenging optimization for deep structural feature capture. Although existing tensor networks alleviate computational burdens through tensor decomposition techniques, most exhibit limited capability in learning non-linear relationships. To overcome these limitations, we introduce the Mode-Aware Non-linear Tucker Autoencoder (MA-NTAE). MA-NTAE generalized classical Tucker decomposition to a non-linear framework and employs a Pick-and-Unfold strategy, facilitating flexible per-mode encoding of high-order tensors via recursive unfold-encode-fold operations, effectively integrating tensor structural priors. Notably, MA-NTAE exhibits linear growth in computational complexity with tensor order and proportional growth with mode dimensions. Extensive experiments demonstrate MA-NTAE’s performance advantages over DAE variants and current tensor networks in dimensionality reduction and recovery, which become increasingly pronounced for higher-order, higher-dimensional tensors.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 10713