Social Hierarchy-Guided Evolutionary Neural Architecture Search for Efficient and Automated Design

Fang Su, YuChen Jing, Yance Wang, Jiajun Yao

Keywords: Evolutionary Computation-based NAS (ENAS), Social Hierarchy Guidance, Progressive Evaluation Search

Abstract: Neural Architecture Search (NAS) serves as an important component in Automated Machine Learning. Compared with reinforcement learning and gradient-based NAS approaches, evolutionary computation-based NAS (ENAS) has gained prominence due to its lower dependence on domain expertise and superior adaptability across diverse problem domains. However, despite a lot of research, how to significantly reduce the computational cost while pursuing high accuracy is still a huge challenge for ENAS. To address this issue, we propose a Social Hierarchy-guided Evolutionary Neural Architecture Search algorithm (SH-ENAS). In this algorithm, inspired by the social hierarchy, a novel population organization structure is designed, and based on it, effective guidance operations are designed for the subsequent evolutionary search process. Additionally, to further reduce computational overhead, a progressive evaluation search method is proposed, which introduces weight inheritance and validation-loss-guided early stopping operation to prevent unnecessary evaluations of the architecture. The experimental results demonstrate that SH-ENAS achieves test errors of $2.50\%$ and $16.24\%$ on CIFAR-10 and CIFAR-100, respectively, outperforming existing state-of-the-art methods. In particular, SH-ENAS requires only $10$ population individuals and $12$ iterations to complete the search, with computational costs as low as $0.69$ GPU days and $0.83$ GPU days, validating the significant advantages of the new algorithm in terms of accuracy, computational efficiency, and automation.

Primary Area: Deep learning (e.g., architectures, generative models, optimization for deep networks, foundation models, LLMs)

Submission Number: 14238