# How to Reproduce the Results of the Numerical Experiments in Sections 7.1, 7.2, 7.3, and D.2.

## Preparation

### Python Environment
You may need to create a virtual environment and install the following packages using pip:

```
python -m venv ./env_dre_expr
source ./env_dre_expr/bin/activate
pip install --upgrade pip
pip install numpy
pip install pandas
pip install matplotlib
pip install -U scikit-learn
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install lightning
pip install tensorboard
pip install lightgbm
pip install wandb
pip install -U skorch
pip install omegaconf
pip install openpyxl
pip install hydra-core
pip install python-box
pip install pytorch-minimize
```

#### 2. Initialize WandB and Log In
Our code uses the ```wandb``` library. 
You will need to log in to your WandB account to run these scripts. 

After installing the library, run the following command to log in:

```
wandb login
```


---

## Running the Experiments

### Reproducing the Results in Section 7.1.

#### Run Scripts
To reproduce all the results of the experiments discussed in Section 7.1, run the following script:

```
source env_dre_expr/bin/activate
python 1-1_run_experiment_Section_7_1_StabilityDuringOptimization.py
```


### Reproducing the Results in Section 7.2.

#### 1. Generate Datasets
First, generate the datasets used in the experiments. The datasets will be saved in the ```data/``` directory:

```
source env_dre_expr/bin/activate
python 1-2_generate_data_Section_7_2_EfficiencyofUnbiasedGradients.py
```

#### 2. Run Scripts
To reproduce all the results of the experiments, run the following script:

```
source env_dre_expr/bin/activate
python 1-2_run_experiment_Section_7_2_EfficiencyofUnbiasedGradients.py
```

s
### Reproducing the Results in Section 7.3.

#### 1. Generate Datasets
First, generate the datasets used in the experiments. The datasets will be saved in the ```data/``` directory:

```
source env_dre_expr/bin/activate
python 1-3_generate_data_Section_7_3_HighKLdivergenceData.py
```

#### 2. Run Scripts
* **For $\alpha$-Div**:  
  To reproduce all the results of $\alpha$-Div, run the following script:

  ```
  source env_dre_expr/bin/activate
  python 1-3_run_experiment_Section_7_3_HighKLdivergenceData-alphaDiv.py
  ```

* **For the KL-divergence loss function**:  
  To reproduce all the results of $\alpha$-Div, run the following script:

  ```
  source env_dre_expr/bin/activate
  python 1-3_run_experiment_Section_7_3_HighKLdivergenceData-KLloss.py
  ```

### Reproducing the Results in Section D.1.
To reproduce the results discussed in Section D.1, read the `README.md` file in the `./1-4_run_experiment_Section_D_1/` folder and follow the procedure outlined in the document.


### Reproducing the Results in Section D.2.
To reproduce all the results of the experiments, run the following script:
#### Run Scripts

```
bash 1-5_run_experiment_Section_D_2_DomatinAdaptaition.sh
```
