TA-GATES: An Encoding Scheme for Neural Network ArchitecturesDownload PDF

Published: 31 Oct 2022, 18:00, Last Modified: 09 Jan 2023, 15:41NeurIPS 2022 AcceptReaders: Everyone
Keywords: Architecture Encoding Scheme, Neural architecture search (NAS), Predictor-based NAS, AutoML
TL;DR: This work proposes a new encoding scheme for neural architectures, the Training-Analogous Graph-based ArchiTecture Encoding Scheme (TA-GATES).
Abstract: Neural architecture search tries to shift the manual design of neural network (NN) architectures to algorithmic design. In these cases, the NN architecture itself can be viewed as data and needs to be modeled. A better modeling could help explore novel architectures automatically and open the black box of automated architecture design. To this end, this work proposes a new encoding scheme for neural architectures, the Training-Analogous Graph-based ArchiTecture Encoding Scheme (TA-GATES). TA-GATES encodes an NN architecture in a way that is analogous to its training. Extensive experiments demonstrate that the flexibility and discriminative power of TA-GATES lead to better modeling of NN architectures. We expect our methodology of explicitly modeling the NN training process to benefit broader automated deep learning systems. The code is available at https://github.com/walkerning/aw_nas.
Supplementary Material: pdf
17 Replies

Loading