SMPE: A Framework for Multi-Dimensional Permutation Equivariance
Primary Area: learning on graphs and other geometries & topologies
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Permutation equivariance, multi-dimensional equivariant network, feature reuse, cross-dimensional information.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: A novel framework that enables cross-dimensional interactions among objects while maintaining multi-dimensional permutation equivariance.
Abstract: Permutation equivariance (PE) is an important inductive prior for addressing tasks such as point cloud segmentation, where permuting objects in the input set maintains the output features of each object. However, the state-of-the-art PE methods mainly focused on one dimensional cases, which cannot meet the requirements of multi-dimensional tasks such as auction design, pseudo inverse computation, and multiuser resource allocation in wireless networks. It is evidenced that the direct incorporation of high-dimensional equivariance in network design necessitates tensor operations and complicated parameter sharing patterns, which contribute to its limited exploration. In this paper, we propose a novel serial multi-dimensional permutation equivariance (SMPE) framework to address these challenges. By serially composing multiple one-dimensional equivariant layers and incorporating dense connections for feature reuse, the proposed SMPE framework enables cross-dimensional interactions among objects while maintaining multi-dimensional equivariance. Additionally, we extend the SMPE framework to scenarios of permutation invariance as well as the hybrid equivariance and invariance through pooling operations. We use an extensive set of experiments to evaluate the framework on contextual auction design, pseudo inverse computation, and multiuser wireless communication tasks. It is observed that the SMPE framework not only maintains excellent equivariance property to support variable set sizes, but also outperforms the state-of-the-art models. For example, SMPE could gain as high as 8.4% and 14.4% improvements over the state-of-the-art methods in two typical multiuser resource allocation scenarios.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 5733
Loading