Equivariant Neural Functional Networks for Transformers

Hoang V. Tran, Thieu Vo, An Nguyen The, Tho Tran Huu, Minh-Khoi Nguyen-Nhat, Thanh Tran, Duy-Tung Pham, Tan Minh Nguyen

Track: long paper (up to 8 pages)

Keywords: neural functional network, transformer, maximal symmetric group, equivariant model, dataset

TL;DR: This paper systematically studies neural functional networks (NFNs) for Transformers, presenting a design principle, and an equivariant NFN called Transformer-NFN, along with a benchmark dataset for evaluation.

Submission Number: 27