############################################################################
#                                                                          #
#                     Code for ICLR'22 submission 2974                     #
#  INFORMATION BOTTLENECK: EXACT ANALYSIS OF (QUANTIZED) NEURAL NETWORKS   #
#                                                                          #
############################################################################

This archieve contains code and instructions for experiments for the paper:
Information Bottleneck: Exact Analysis of (quantized) neural networks


#################
# Requirements: #
#################
The code is tested with Python 3.8.7 in Arch Linux. The following libraries
are required:

 - Tensorflow 2.5.0
 - Numpy
 - Pandas
 - Matplotlib


##################
# Code structure #
##################
The main part of the code is located in the IB directory/module. Important
files are:

 IB/models/models.py  # Contains definition of the models from the experiments
 IB/util/estimator.py # Contains the MI estimators/computation
                        (binning and quantized)
 IB/data/var_u.mat    # The synthetic data set
 IB/experiment.py     # Code running the actual experiment

Experiment code is included in the experiment directory. Subdirectories 
includes latex code for creating the plots from the paper.
Important files are:

 experiment/Makefile  # Used to run experiments and create plots
 helpers/quantize.py  # Helper for running quantization experiments
 helpers/binning.py   # Helper for running binning experiments


#######################
# Running experiments #
#######################  
The Makefile in the experiments directory is provided for running the
experiments from the paper and generating plots. Note, that the experiments
and plot generating may take a file to finish.

To run experiments, cd to the experiments directory. Before starting
the experiments, run "make link" and "make data". This links the IB module
and downloads MNIST.

The following targets are available in the Makefile:

 SETUP:
 - make link             # Links the IB module
 - make data             # Downloads MNIST
 EXPERIMENTS:
 - make quantize         # Runs the quantization experiments included in the
                           body of the paper. Outputs are stored in out/.
 - make quantize-extra   # Runs the quantization experiments included in the
                           appendix. Outputs are stored in out/.
 - make binning          # Runs the binning experiments included in the
                           appendix. Outputs are stored in out/.
 PLOTS:
 - make plot-quantize          # Generates the plots from the body of the
                                 paper. Outputs are stored in plots/.
 - make plot-quantize-extra    # Generates the quantization plots from the
                                 appendix. Outputs are stored in plots/.
 - make plot-binning           # Generates the binning plots from the
                                 appendix. Outputs are stored in plots/.
 - make plot-accuracy          # Generates the accuarcy plots from the
                                 appendix. Outputs are stored in plots/.
