# Regulation Games for Trustworthy Machine Learning

## Structure
The repository contains the following parts:
    * Code to run an instance of ParetoPlay game
    * Code to train a FairPATE student model for calibration
    * Notebook with functions to graph ParetoPlay game results

### ParetoPlay Game Instance
`experiments/` has script that runs experiments on CelebA, FairFace, UTKFace using FairPATE and on MNIST using DPSGD-Global-Adpt.
`parameters.py` contains all game parameters and calibration parameters. 
`main.py` contains code to start an experiment, which includes running a game simulation, saving game parameters and results, and plotting the results. 
`simulation.py` contains code for running the game.
`agents.py` contains code of all game agents. 

### Calibration
`calibration.py` contains functions that run on different algorithm baselines and these functions are called during calibration to train a model instance.
The baselines algorithms we use are stored in `baselines/`.
FairPATE: 
`fairpate\` stores all code used for FairPATE.

## Experiment
The `main.py` file contains the starting point. The parameters for the program
are in `parameters.py` file.

### Examples of the pipeline:

The bash code for running the scripts can be found in the `experiments` folder.

```
EROOT= [path that points to the directory] \
RROOT= [path that points to where intermediate models should be stored] \

poetry run python $EROOT/main.py \
    --experiment_name [Name of the experiment for creating an experiment folder] \
    --prev_results_dir $EROOT/previous_results/ \
    --save_path $EROOT/results/ \
    --path $RROOT/ \
    --data_dir [path that points to the dataset] \
    --num_rounds [Number of game rounds] \
    --init_priv [Initial privacy specification] \
    --init_fair [Initial fairness specification] \
    --dataset [Dataset name] \
    --priority [Agent that moves first] \
    --step_size_priv [Privacy parameter step size] \
    --step_size_fair [Fairness parameter step size] \
    --step_size_decay [Step size decay factor] \
    --builder_lambda [Model builder lambda when weighing its own loss. Used in fairPATE to weight accuracy vs coverage] \
    --lambda_priv [Lambda priavcy] \
    --lambda_fair [Lambda fairness] \
    --C_priv [Privacy penalty factor] \
    --C_fair [Fairness penalty factor] \
    --goal_priv [Privacy regulator's constraint] \
    --goal_fair [Fairness regulator's constraint] \
    --calibration [Whether to include calibration round]
```

## Environment 
This project uses [Poetry](https://python-poetry.org) for dependency management. To start developing, first [install `poetry`](https://python-poetry.org/docs/master/#installing-with-the-official-installer) then:

```
>> cd .
>> poetry install
```

Poetry will install all dependencies and create an appropriate virtual environment for the project. To get started with a notebook server: 

```
>> poetry run jupyter lab
```

Otherwise, to run python scripts in the environment:
```
>> poetry run python <script>.py
```

For example:
```
>> poetry run bash experiments/utkface_experiment.sh
```

To get the location of the environment:
```
>> poetry run which python
/Users/example/Library/Caches/pypoetry/virtualenvs/governance-games-S7sOSjaR-py3.9/bin/python
```


