Evolutionary Neural Architecture Search for Transferable Networks

Xun Zhou, Songbai Liu, A. Kai Qin, Kay Chen Tan

Abstract: The recent proliferation of edge computing has led to the deployment of deep neural networks (DNNs) on edge devices like smartphones and IoT devices to serve end users. However, developing the most suitable DNN model for each on-device task is nontrivial, due to data governance of these tasks and data heterogeneity across them. Existing approaches tackle this issue by learning task-specific models on the device, but this requires substantial computational resources, exacerbating the computational and energy demands on edge devices. This research strives to enhance the deployment efficiency of advanced models on edge devices, with a specific focus on reducing the on-device learning cost. In pursuit of this goal, we propose a category-specific but task-agnostic evolutionary neural architecture search (CSTA-ENAS) method. This method can utilize the available datasets from multiple other tasks in the same category as on-device tasks to design a transferable architecture on the server. Then, this architecture only requires light on-device fine-tuning to satisfactorily solve all different on-device tasks, significantly reducing the on-device learning time and related energy consumption. To improve the search efficiency of our method, a supernet-based partial training strategy is proposed to reduce the evaluation cost for candidate architectures. To showcase the effectiveness of CSTA-ENAS, we build transferable DNN models and evaluate their accuracies on a set of new image classification tasks. Our models demonstrate competitive performance compared to most of the existing task-specific models and transferable models while requiring fewer on-device computational resources.

External IDs:dblp:journals/tetci/ZhouLQT25