From One to Zero: Causal Zero-Shot Neural Architecture Search by Intrinsic One-Shot Interventional Information

Qian Li, Chao Ma, Zhengqin Xu, Xiaokang Yang

Keywords: Neural Architecture Search

Abstract:

''Zero-shot'' neural architecture search (ZNAS) is key to achieving real-time neural architecture search. ZNAS comes from ''one-shot'' neural architecture search but searches in a weight-agnostic supernet and consequently largely reduce the search cost.
However, the weight parameters are agnostic in the zero-shot NAS and none of the previous methods try to explain it.
We question whether there exists a way to unify the one-shot and zero-shot experiences for interpreting the agnostic weight messages. To answer this question, we propose a causal definition for ''zero-shot NAS'' and facilitate it with interventional data from ''one-shot'' knowledge.
The experiments on the standard NAS-bench-201 and CIFAR-10 benchmarks demonstrate a breakthrough of search cost which requires merely 8 GPU seconds on CIFAR-10 while maintaining competitive precision.

Submission Number: 1624