# Instructions to replicate experiments in the paper "Designing ethical environments using multi-agent reinforcement learning"

## Install EPyMARL dependencies
Follow the instructions in the epymarl_README.md file included in the supplementary material to install EPyMARL dependencies.
It is recommended to create a new virtual environment to install the dependencies.
## Install the Ethical Gathering Game
Within the same virtual environment, install the Ethical Gathering Game by running the following command:
```bash
pip install MAEGG.tar.gz
```
The tar file can be found in the supplementary material.

## Run experiments
Within the same virtual environment, run the following command to train the agents in the Ethical Gathering Game:
To run MALPPO run to get reference policies run the following command:
```bash
python src/main.py --env-config=egg_large_mo --config=ippo_egg_mo with env_args.efficiency=0.4
```
To run MAPPO for a specific scalarised environmet with ``` w_e = 2.5``` and efficient population set to 40 run the following command:
```bash
python src/main.py --env-config=egg_large --config=mappo_egg with env_args.we=2.5 env_args.efficiency=0.4
```

To run IPPO for a specific scalarised environment with ``` w_e = 2.5``` and efficient population set to 40 run the following command:
```bash
python src/main.py --env-config=egg_large --config=ippo_egg with env_args.we=2.5 env_args.efficiency=0.4
```

Note that for --env-config there are two options ```egg_large``` and ```egg_large_mo``` which correspond to the scalarised environment and the multi-objective environment respectively. 

## Evaluate the policies
To evaluate the policies run the following command:
```bash
python src/test_policies.py --model path/to/model --eff 0.8 --config {ippo_egg_mo/mappo_egg/ippo_ns}
```
This will generate a txt file and a png. Text file contains the value vectors and statistics of the EGG, and the image has the median number of apples of each agent throughout the simulation. 

## Extra training parameters
For changing extra parameters of the training, check the original EPyMARL repository for more information. 
```
