Sequence Alignment-based Similarity Metric in Evolutionary Neural Architecture Search
Keywords: Evolutionary NAS, Similarity Metric, Diversity Control
TL;DR: Our paper presents a novel similarity metric based on sequence alignment to control diversity (exploration vs. exploitation) in evolutionary neural architecture search.
Abstract: Neural Architecture Search (NAS) has emerged as a powerful method for automating the design of deep neural networks across diverse applications, with evolutionary optimization showing particular promise in addressing its intricate demands. However, the effectiveness of this approach highly depends on balancing exploration and exploitation, ensuring that the search does not prematurely converge to suboptimal solutions while still achieving near-optimal outcomes. This paper addresses this challenge by proposing a novel similarity metric inspired by global sequence alignment from biology. Unlike most of the existing methods that require pre-trained models for comparison, our metric operates directly on neural network architectures within the defined search space, eliminating the need for model training. We outline the computation of the normalized similarity metric and demonstrate its application in quantifying diversity within populations in evolutionary NAS. Experimental results conducted on popular datasets for image classification, such as CIFAR-10, CIFAR-100, and ImageNet16-120, show the effectiveness of our approach in guiding diversity based on our suggested control function. Additionally, we highlight the usefulness of our similarity metric in comparing individuals to identify advantageous or disadvantageous architectural design choices. The code is available at \url{https://anon-github.automl.cc/r/evonas_similarity_metric-1BD3/}.
Submission Checklist: Yes
Broader Impact Statement: Yes
Paper Availability And License: Yes
Code Of Conduct: Yes
Optional Meta-Data For Green-AutoML: This blue field is just for structuring purposes and cannot be filled.
CPU Hours: 75
GPU Hours: 750
TPU Hours: 0
Evaluation Metrics: No
Submission Number: 37
Loading