SNAP: Generalizable Zero-Shot Prediction of Neural Architecture Performance via Semantic Embedding and Graph Learning

Ye Qiao, Jingcheng Li, Haocheng Xu, Sitao Huang

Keywords: Neural Architecture Search, Zero-Shot Prediction, Graph Learning

Abstract: Neural Architecture Search (NAS) is a powerful approach to discovering high-performing CNN architectures, but most existing methods incur significant computational costs due to extensive training or sampling. Zero-shot NAS predictors offer an efficient alternative by predicting architecture performance without additional training. However, current methods often yield suboptimal predictions—frequently outperformed by basic metrics like parameter counts or FLOPs—and struggle to generalize across different search spaces or unseen operators. To address these limitations, we propose SNAP(Semantic Neural Architecture Predictor), a novel zero-shot neural predictor that leverages a transformer-based semantic embedding of operator descriptions combined with a Graph Convolutional Network (GCN) for architecture performance prediction. Unlike traditional model-based predictors, SNAP requires only a single initial training phase on NASBench-101, after which it effectively generalizes to arbitrary new search spaces and previously unseen operators without fine-tuning. Extensive experiments across diverse NAS benchmarks demonstrate SNAP’s state-of-the-art rank correlation and superior generalization capabilities. Furthermore, SNAP achieves more than 35$\times$ search efficiency improvements, discovering competitive architectures with 93.75\% CIFAR-10 accuracy on NAS-Bench-201 and 74.9\% ImageNet top-1 accuracy on the DARTS space, positioning it as a robust and generalizable foundation for efficient neural architecture search.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 9767