This directory contains all the code needed to reproduce the experiments from
our paper. You'll need a python 3 installation with numpy, picos, and tqdm
installed.
Many of our experiments are run / analyzed using Ipython notebooks.
Our plots are made using matplotlib.

All scripts / IPython notebooks needed are in the experiments/ directory.

Figure 1: Use introExample.ipynb. You'll have to download the ``wine'' and
   ``life expectancy'' datasets that are linked in the paper.
   
Figure 2: Use compareLocalMinQuality_CVObjective.ipynb (this one is actually just fun to
    mess around with to see what patterns you can make)

Figure 3 top left: Use howCloseToUniform.ipynb. To remake this experiment with
   error bars, use howCloseToUniform-withErrorbars.ipynb

Figure 3 top right and bottom: Use Uassumption.ipynb. To remake this experiment
   with error bars, use Uassumption-withErrorbars.ipynb.

Figure 4: This one takes much longer to run. The work to produce the data is
   in epsilonScalingExperiment_parallelN.py. In there, you'll want to adjust
   the variable nCores to be set to how many cores you want to dedicate to
   running this experiment. You should create the directory experiments/output
   ahead of time, as this is where things are saved. Note that the output can be
   pretty large; if you have limited RAM or harddrive space, we recommend
   saving only the "meanQVX" version (the block of code beginning on Line 76);
   all you need to reproduce our plots is nonQVX.mean((2,3)) (the fraction of
   non-QVX examples over all Ehats and U's), and this block of code only
   saves nonQVX.mean((2,3)), which saves a lot of space.

   To plot the results, use vizBigEpsilonExperiment.ipynb


Figures 5 and 6: These figures are produced by:
            compareLocalMinQuality_GradientDescent.ipynb
            compareLocalMinQuality_GridSearch.ipynb
            compareLocalMinQuality_KFoldCV.ipynb
            compareLocalMinQuality_TestObjective.ipynb
       Each notebook contains code to plot the results. To produce the exact
        plots in our paper, save the output and then run 

