# CoCoFL

## Installation & Preparation
* Install requirements with `python3 -m pip install -r requirements.txt`  
* For the FEMNIST/XCHEST/IMDB/Shakespeare experiments, it is required to download the data folder (anonymized link https://drive.google.com/file/d/16qMvuDLmhUBz2ufcSEaPNubN0TZTCkuO/view?usp=sharing) and place it into the repository.
* For running the NN profiling on ARM, it is required to build an ARM compiled torch wheel and place the wheel into the `docker` folder
### Preparation for Profiling (`x64`, `arm`)
* `cd docker`
* `make x64`
* `cd ..`
## Run Experiments

MobileNet experiments With different seeds (e.g., 10,12,13) and algorithms
* `python3 run_fl.py --path settings/fl_setting_mobilenet.json --param algorithm CoCoFL model QMobileNet seed 10`
* `python3 run_fl.py --path settings/fl_setting_mobilenet.json --param algorithm FedAvg model MobileNet seed 10`
* `python3 run_fl.py --path settings/fl_setting_mobilenet.json --param algorithm FedAvgDropDevices model MobileNet seed 10`
* `python3 run_fl.py --path settings/fl_setting_mobilenet.json --param algorithm FjORD model SMobileNet seed 10`
* `python3 run_fl.py --path settings/fl_setting_mobilenet.json --param algorithm HeteroFL model SMobileNet seed 10`


MobileNetLarge (XChest)
* `python3 run_fl.py --path settings/fl_setting_mobilenet_large.json --param algorithm CoCoFL model QMobileNetLarge seed 10`
* `python3 run_fl.py --path settings/fl_setting_mobilenet_large.json --param algorithm FedAvg model MobileNetLarge seed 10`
* `python3 run_fl.py --path settings/fl_setting_mobilenet_large.json --param algorithm FedAvgDropDevices model MobileNetLarge seed 10`
* `python3 run_fl.py --path settings/fl_setting_mobilenet_large.json --param algorithm FjORD model SMobileNetLarge seed 10`
* `python3 run_fl.py --path settings/fl_setting_mobilenet._largejson --param algorithm HeteroFL model SMobileNetLarge seed 10`

ResNet18 Experiments (requires FEMNIST data)
* `python3 run_fl.py --path settings/fl_setting_resnet18.json --param algorithm CoCoFL model QResNet18 seed 10`
* `python3 run_fl.py --path settings/fl_setting_resnet18.json --param algorithm FedAvg model ResNet18 seed 10`
* `python3 run_fl.py --path settings/fl_setting_resnet18.json --param algorithm FedAvgDropDevices model ResNet18 seed 10`
* `python3 run_fl.py --path settings/fl_setting_resnet18.json --param algorithm FjORD model SResNet18 seed 10`
* `python3 run_fl.py --path settings/fl_setting_resnet18.json --param algorithm HeteroFL model SResNet18 seed 10`

ResNet50 Experiments (CIFAR100)
* `python3 run_fl.py --path settings/fl_setting_resnet50.json --param algorithm CoCoFL model QResNet50 seed 10`
* `python3 run_fl.py --path settings/fl_setting_resnet50.json --param algorithm FedAvg model ResNet50 seed 10`
* `python3 run_fl.py --path settings/fl_setting_resnet50.json --param algorithm FedAvgDropDevices model ResNet50 seed 10`
* `python3 run_fl.py --path settings/fl_setting_resnet50.json --param algorithm FjORD model SResNet50 seed 10`
* `python3 run_fl.py --path settings/fl_setting_resnet50.json --param algorithm HeteroFL model SResNet50 seed 10`

DenseNet Experiments (CIFAR10)
* `python3 run_fl.py --path settings/fl_setting_densenet.json --param algorithm CoCoFL model QDenseNet seed 10`
* `python3 run_fl.py --path settings/fl_setting_densenet.json --param algorithm FedAvg model DenseNet seed 10`
* `python3 run_fl.py --path settings/fl_setting_densenet.json --param algorithm FedAvgDropDevices model DenseNet seed 10`
* `python3 run_fl.py --path settings/fl_setting_densenet.json --param algorithm FjORD model SDenseNet seed 10`
* `python3 run_fl.py --path settings/fl_setting_densenet.json --param algorithm HeteroFL model SDenseNet seed 10`

Repeat with additional parameter `dataset CINIC10` for CINIC10

Transformer Experiments (IMDB)
* `python3 run_fl.py --path settings/fl_setting_transformer.json --param algorithm CoCoFL model QTransformer seed 10`
* `python3 run_fl.py --path settings/fl_setting_transformer.json --param algorithm FedAvg model Transformer seed 10`
* `python3 run_fl.py --path settings/fl_setting_transformer.json --param algorithm FedAvgDropDevices model Transformer seed 10`
* `python3 run_fl.py --path settings/fl_setting_transformer.json --param algorithm FjORD model STransformer seed 10`
* `python3 run_fl.py --path settings/fl_setting_transformer.json --param algorithm HeteroFL model STransformer seed 10`



TransformerSeq2Seq Experiments (Shakespeare)
* `python3 run_fl.py --path settings/fl_setting_transformer_seq2seq.json --param algorithm CoCoFL model QTransformerSeq2Seq seed 10`
* `python3 run_fl.py --path settings/fl_setting_transformer_seq2seq.json --param algorithm FedAvg model TransformerSeq2Seq seed 10`
* `python3 run_fl.py --path settings/fl_setting_transformer_seq2seq.json --param algorithm FedAvgDropDevices model TransformerSeq2Seq seed 10`
* `python3 run_fl.py --path settings/fl_setting_transformer_seq2seq.json --param algorithm FjORD model STransformerSeq2Seq seed 10`
* `python3 run_fl.py --path settings/fl_setting_transformer_seq2seq.json --param algorithm HeteroFL model STransformerSeq2Seq seed 10`


Ablation Experiments for different values of l (model can be `QMobileNet`, `FMobileNet`, `MobileNet`, `QMobileNet_ablation`, `FMobileNet_ablation`). The programm asserts if no configuration that satisfies the constraint is found:
* `python3 run_fl.py --path settings/fl_setting_ablation_study.json --param model QMobileNet --param param_round_resources ablation__1.0 seed 10`
* `python3 run_fl.py --path settings/fl_setting_ablation_study.json --param model QMobileNet --param param_round_resources ablation__0.8 seed 10`
* `python3 run_fl.py --path settings/fl_setting_ablation_study.json --param model QMobileNet --param param_round_resources ablation__0.6 seed 10`

For non-iid or rc-non-iid versions add `dataset noniid_CIFAR10` or `dataset noniid_resources_CIFAR10`. Same applies for CINIC10, CIFAR100, FEMNIST, XCHEST, SHAKESPEARE  

## Run Profiling
* `./python3-profiling-x64 profile.py --algorithm CoCoFL --model QResNet18 --architecture arm` (make sure to run on an ARM device)
* `./python3-profiling-x64 profile.py --algorithm CoCoFL --model QResNet50 --architecture x64`
* `./python3-profiling-x64 profile.py --algorithm CoCoFL --model QMobileNet --architecture x64`
* `./python3-profiling-x64 profile.py --algorithm CoCoFL --model QDenseNet --architecture arm` (make sure to run on an ARM device)
* `./python3-profiling-x64 profile.py --algorithm CoCoFL --model QTransformer --architecture x64`  


* `./python3-profiling-x64 profile.py --algorithm Subset --model SResNet18 --architecture arm` (make sure to run on an ARM device)
* `./python3-profiling-x64 profile.py --algorithm Subset --model SResNet50 --architecture x64`
* `./python3-profiling-x64 profile.py --algorithm Subset --model SMobileNet --architecture x64`
* `./python3-profiling-x64 profile.py --algorithm Subset --model SDenseNet --architecture arm` (make sure to run on an ARM device)
* `./python3-profiling-x64 profile.py --algorithm CoCoFL --model STransformer --architecture x64`
* `./python3-profiling-x64 profile.py --algorithm CoCoFL --model STransformerSeq2Seq --architecture x64`


This creates `json` files in the folder `table`. The tables used for the experiments are already put in `nets/QuantizedNets/{NN-structure}/tables/` and `nets/SubsetNets/{NN-structure}/tables/`
