We implemented and performed most of the numerical experiments with python. But we prepared the plot in Matlab from the data we collected from these experiments. To replicate the experiments, you can follow the following steps:

- Section 6.1
Run "./Section6_1/main_*.sh" files to run the experiments. This will save the results into the folder "./Section6_1/runs". Then, you can run "./Section6_1/plotFigure1.m" to generate Figure 1 and "./Section6_1/plot_*.m" to generate Figure S.1 in the supplementary report.

- Section 6.2
Run "./Section6_2/experiment.py". This will save results into the folder "./Section6_2/runs". Then, you can run "./Section6_2/plotFigure2.m" to generate Figure 2 and "./Section6_2/plotImages.m" to generate Figure S.2 in the supplementary report.

- Section 6.3
Run "./Section6_3/run_mnist_seeds.sh". This will save results into the folder "./Section6_3/runs". Then, run "./Section6_3/runs/auto_pickle_to_mat_converter.py". This will convert pickle data to ".mat" file so that we can load it to MATLAB. Finally, you can run "./Section6_3/plotFigure3.m" to generate Figure 3. 

This folder also contains the additional files ("cifar-10.py" and "run_experiments_cifar.py") for our preliminary experiments with the ResNet model. These codes assume cifar dataset from "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz" is downloaded and extracted to "./Section6_3/cifar-10-batches-py".

- Section G.1
This experiment is performed in Matlab R2018a. To replicate the results, run "./SectionG_1/main_isotonic_regression.m". This will save results into the folder "./SectionG_1/runs". Then, you can run "./SectionG_1/plot_isotonic_regression.m" to generate Figure S.8. 

- Section G.2
This experiment is performed in Matlab R2018a. To replicate the results, first you need to download the datasets from "http://www.cs.technion.ac.il/~rani/portfolios/" to the folder "./SectionG_2/data". Then, run "./SectionG_2/main_portfolio_optimization.m". This will save results into the folder "./SectionG_2/runs". Then, you can run "./SectionG_2/plot_portfolio_optimization.m" to generate Figure S.9. 


If you need help, please contact gua@mit.edu or alp.yurtsever@umu.se