NEURAL ADDITIVE TENSOR DECOMPOSITION FOR SPARSE TENSORS

Dawon Ahn, Uday Singh Saini, Evangelos E. Papalexakis, Ali Payani

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Tensor Decomposition, Neural Tensor Models, Interpretability

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

Canonical Polyadic Decomposition (CPD) is a fundamental technique for tensor analysis, discovering underlying multi-linear structures represented as rank-one tensors (components). The simplicity of the rank-one tensors facilitates the interpretation of hidden structures within tensors compared to other types of conventional tensor decomposition models. However, CPD has limitations in modeling nonlinear structures present in real-world tensors. Recent tensor decomposition models combined with neural networks have shown superior performance in tensor completion tasks compared to multi-linear tensor models. Nevertheless, one drawback of those nonlinear tensor models is the lack of interpretability since their black-box approaches entangle all interactions between latent components, unlike CPD, which handles the components individually as rank-one tensors.

To overcome this major limitation and bridge the gap between CPD and various state-of-the-art neural tensor models,
we propose Neural Additive Tensor Decomposition (NeAT) to accurately capture non-linear interactions in sparse tensors while respecting the separation of distinct components in a similar vein as CPD. The main idea is to neuralize each component to model non-linear interactions within each component separately. This not only captures non-linear interactions but also makes the decomposition results easy to interpret by being as close to the CPD model as possible. Extensive experiments with six large-scale real-world datasets demonstrate that \method{} is more accurate than the state-of-the-art neural tensor models and easy to interpret latent patterns.
In the link prediction task,
NeAT outperforms CPD by 10\% and the second-best performing neural tensor model by 4\%, in terms of AUC score.
Finally, we demonstrate the interpretability of NeAT by visualizing and analyzing latent components from real data.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8136