# Is Differentiable Architecture Search truly a One-Shot Method?

This is code for the AutoML submission "Differentiable Architecture Search: a One-Shot Method?" #25

### Abstract:
Differentiable architecture search (DAS) is a widely researched tool for the design of novel architectures. The main benefit of DAS is the effectiveness achieved through the weight-sharing one-shot paradigm, which allows efficient architecture search. In this work, we investigate DAS in a systematic case study of inverse problems, which allows us to analyze these potential benefits in a controlled manner. We demonstrate that the success of DAS can be extended from image classification to signal reconstruction, in principle. However, our experiments also expose three fundamental difficulties in the evaluation of DAS-based methods in inverse problems: First, the results show a large variance in all test cases. Second, the final performance is strongly dependent on the hyperparameters of the optimizer. And third, the performance of the weight-sharing architecture used during training does not reflect the final performance of the found architecture well. While the results on image reconstruction confirm the potential of the DAS paradigm, they challenge the common understanding of DAS as a one-shot method.

### Requirements:
- PyTorch >= 1.6
- hpbandster [can install via pip]

### Replication:
Run the scripts starting with the tag `automl_` to replicate our main experiments.
Each script contains information about which experiments (tables and figures) are reproduced. 
To rerun the hyperparameter optimization, run the scripts starting with the tag `bohb_`. 



#### In more detail:
Code for the experiments in Sections 4.2 and 4.3 on the Blur data formation for the sequential search space from Section 3.1.1:
bash automl_Blur_sequential_sec_4_2_4_3.sh


Code for the experiments in Sections 4.2 and 4.3 on the Downsampling data formation for the sequential search space from Section 3.1.1:
bash automl_Downsampling_sequential_sec_4_2_4_3.sh

Code for the experiments in Section 4.2 on the Blur data formation for the non-sequential search space from Section 3.1.2:
bash automl_Blur_non_sequential_sec_4_2


Code for the experiments in Section 4.4 for image reconstruction (Table 4):
bash automl_BSDS_Blur_sequential_sec_4_4.sh

Code for the experiments in Appendix Section B for the Blur data formation for the sequential search space from Section 3.1.1:
bash automl_Blur_sequential_initialization_sec_B.sh





