# CRAFTING ZERO-COST PROXY METRICS FOR NEURAL ARCHITECTURE SEARCH VIA SYMBOLIC REGRESSION

## Setup

```
$ conda create -n py39 python=3.9
$ conda activate py39
$ pip install -r requirements.txt
```
All numerical results and figures are stored in folders `result` and `fig`, respectively.

## Reproducing the results
You can search from scratch to find 31 expression trees as in the paper or utilize the provided trees in the folder `exp`. 
To search for expression trees, run the below script (all expression trees would be stored in the folder `exp`):
```shell
$ python gp_search_multiple.py --SR_dataset D
```
After the process is finished, we will obtain the Table 2 in the paper.

Whenever we have all expression trees (in case you re-conduct the searches), we can reproduce all results in the paper.
- Figure 5:
```shell
$ python exp_statistics.py
```

- Get the best metric (Equation 2) & Figure 6:
```shell
$ python evaluate_SR-designed_metric.py --measure kendall
```

- Appendix A.1, A.2:
```shell
$ python gp_search_multiple.py --SR_dataset D+
$ python gp_search_multiple.py --SR_dataset D-
$ python gp_search_single.py --ss nb101
$ python gp_search_single.py --ss nb201
$ python gp_search_single.py --ss nb301
$ python run_appendixA.py
```

- Appendix F (all results are stored in the folder `results`):
```shell
$ python run_appendixF.py
```

- Appendix G (all figures are stored in the folder `fig`):
```shell
$ python run_appendixG.py
```