# Discovering Model Structure of Dynamical Systems with Combinatorial Bayesian Optimization

As part of the submission at the Transactions in Machine Learning Research (TMLR) journal, this repository contains the code needed for running all the benchmark problems.

The main optimizer function can be found at [CBO/optimizers/CBO.py](CBO/optimizers/CBO.py).

It implements the surrogate models designed for combinatorial spaces and the acquisition function `FRCHEI` that can handle inequality and crash constraints.

![CBO/tutorial/FRCHEI.png](CBO/tutorial/FRCHEI.png)
(generated with `CBO/tutorial/FRCHEI.ipynb`)

## Installation

Clone this repository with the submodule flags:
```sh
git clone @GUTHUBSSHURL --recurse-submodules
```

If you happen to clone this repository without the `--recurse-submodules` flag, please download the submodules manually as follows:
```sh
git submodule update --force --recursive --init --remote
```

Next, create a virtual environment with python version `3.10.0` and install the dependencies:
1. install the `CBO` deppendencies:
    ```sh
    cd CBO
    pip install -e .
    ```

2. install the `PR` deppendencies:
    ```sh
    cd bo_pr
    pip install -e .
    ```

## Testing

Python unit tests available at `CBO/test/`

## Running Benchmark Experiments

The main script for running equation discovery experiments is [run_equation_discovery.py](run_equation_discovery.py). See the help for more information on how to run it:
```sh
./run_equation_discovery.py --help
```

To reproduce the experiments as in the paper run the following scripts, switching the `experiment_name` variable to the desired experiment:
```sh
# get current timestamp
timestamp=$(date +"%Y%m%d-%H%M%S")

# experiment_name="NonLinearDampedOscillator_k5"
# experiment_name="Lorenz_k3"
# experiment_name="SEIR_k3"
experiment_name="CylinderWake_k3"
number_of_reruns=10


# ============ Main experiments ============ #
python3 run_equation_discovery.py -e ${experiment_name} -j 1 -r ${number_of_reruns} -m CBO -f FRCHEI_KPOLYDIFF_BS2 -i "${timestamp}"
python3 run_equation_discovery.py -e ${experiment_name} -j 1 -r ${number_of_reruns} -m CBO -f CHEI_KPOLYDIFF_BS2 -i "${timestamp}"
python3 run_equation_discovery.py -e ${experiment_name} -j 1 -r ${number_of_reruns} -m CBO -f FRCEI_KPOLYDIFF_BS2 -i "${timestamp}"

# ============ Ablation study: batch size ============ #
python3 run_equation_discovery.py -e ${experiment_name} -j 1 -r ${number_of_reruns} -m CBO -f FRCHEI_KPOLYDIFF_BS1 -i "${timestamp}"
python3 run_equation_discovery.py -e ${experiment_name} -j 1 -r ${number_of_reruns} -m CBO -f FRCHEI_KPOLYDIFF_BS4 -i "${timestamp}"

# ============ Ablation study: kernel type ============ #
python3 run_equation_discovery.py -e ${experiment_name} -j 1 -r ${number_of_reruns} -m CBO -f FRCHEI_KDIFF_BS2 -i "${timestamp}"
python3 run_equation_discovery.py -e ${experiment_name} -j 1 -r ${number_of_reruns} -m CBO -f FRCHEI_KPOLY_BS2 -i "${timestamp}"

# ============ Baselines ============ #
python3 run_equation_discovery.py -e ${experiment_name} -j 1 -r ${number_of_reruns} -m RS -i "${timestamp}"
python3 run_equation_discovery.py -e ${experiment_name} -j 1 -r ${number_of_reruns} -m SA -i "${timestamp}"
python3 run_equation_discovery.py -e ${experiment_name} -j 1 -r ${number_of_reruns} -m PR -i "${timestamp}"
```

Those will generate result files in the `results/<experiment_name>/main/` folder.

The general experiment configurations can be changed here: [configs_equation_discovery.yaml](configs_equation_discovery.yaml).


## Generating Benchmark Result Figures

After running the experiments you can generate the figures showing the optimization progress by running the following script:
```sh
python3 utils/plot__y_min_feas__vs__iter.py
```
The result pdf images will be saved to the `results` folder, for instance, `results/yminfeas_wall_nbrfeas__x__iter__main.pdf`: 

![yminfeas_wall_nbrfeas__x__iter__main](results/yminfeas_wall_nbrfeas__x__iter__main.png)


