Neural Architecture Generation via Contrastive Representation Learning

Jingrong Xie, Yuqi Feng, Minxiao Zhong, Yanan Sun

Abstract: Neural architecture search aims to automatically design architectures, attracting increasing attention recently. However, these methods need to individually evaluate numerous candidate architectures, thus causing a huge budget of time and computational resources. To address this issue, we propose a novel neural architecture generation method to generate optimal architectures without iterative evaluation and additional training. Specifically, in order to obtain a flexible and comprehensive representation of architectures, we propose a contrastive learning method to capture both static attributions and latent performance properties. Subsequently, a self-guided diffusion model is designed to learn these representations and generate optimal architectures for diverse tasks. Compared with existing methods, our method not only saves the computational time required for evaluating architectures but also gains better performance in different cases. For example, we only need 0.02 GPU Days to generate architecture with 77.1% top-1 accuracy on ImagNet and 97.65% on CIFAR-10. Furthermore, we can find competitive architecture from convolutional and transformer search spaces, i.e., DARTS, NAS-Bench-201, and AutoFormer, which demonstrates the broad applicability of the proposed method.

External IDs:dblp:conf/ijcnn/XieFZS25