# Config Options

The following overview should help to keep track of all available options in the *path learning code framework*.
Exemplar configs can be found in `{DIR_TO_REPO}/configs`. Below the main structure of a config is shown, 
to refer to in which part, which options belongs and some comments are added. It will not work as such! Note furthermore, 
that this file focuses on all available options that change the learning behaviour and not the normally required 
settings such as the number of outputs. How to run an experiment is explained in more detail in `{DIR_TO_REPO}/README.md`.
An explanation to important components, such as a task or the controller, can be found there as well. 

```
"name": "Test",                 
# name of the experiment used for the logs and the google sheet
  "seeds": [101,102],           
# random seeds for which the experiment should be run
  "experiment": {               
    "learn_mapping": {
      "model": {},
      "optimizer": {},
      "controller": {}
    },
    "task_path": {
      "defaults": {
        "domain_defaults": {},
        "loss": {}
      },
      "tasks":[
        {},
        {}
      ]
    },
    "save_model": true          
# save the model after each task
  }
}
```

## "model"

Various models for synthetic ("toy") experiments as well as for visual classification tasks are provided.
For more details refer to `{DIR_TO_REPO}/path_learning/models`.


**Models for synthetic experiments:**
```
"basic_net", "toy_relu_net", "toy_res_net", "toy_svm", "toy_svm_poly", "toy_gaussian_naive",
"toy_batch_relu_net", "toy_batch_res_net", "toy_batch_net", "toy_long_batch_relu_net", "toy_long_batch_res_net"
```

**Models for visual classification experiments:**
```
"resnet", "resnet34", "densenet", "squeezenet", "vgg", "alexnet", "allconv", "clp_mnist_fcn"
```
For all models, except ```"clp_mnist_fcn"``` and ```"allconv"```, one can use pretrained models (```"use_pretrained": true```).

For ```"resnet"``` and ```"resnet34"``` one can use leaky ReLU by adding the slope option (```"slope": 0.1```).

For all models one can apply the bias resetting method. Bias resetting means that before training on a task all network biases 
 are reset by adding a user-specified value (0.01 in the following examples). There are three options: 
 Bias resetting is applied before every task (```"use_bias_reset": 0.01```), 
 the biases are only reset before the very first task, e.i. before any training (```"only_first_bias_reset": 0.01```), 
 or the biases are reset before all tasks except the very first task (```"wo_first_bias_reset": 0.01```). 


In the following an exemplar extract of a config is given:
```
"model": {
 "name": "resnet",
 "n_outputs": 10,
# number of classes
 "slope": 0.1,
 "use_pretrained": true,
 "wo_first_bias_reset": 0.01
}
```

## "controller"

One can introduce a learning rate decay, whereas the rate (e.g. 0.1) by which the learning rate decays needs to be specified as well as
the number of epochs (e.g. 30) after which the decay is applied again (```"lr_decay": {"decay_rate": 0.1, "decay_every_n_epochs": 30}```). 

In the following an exemplar extract of a config is given:
```
"controller": {
  "max_n_epochs": 100,                   
# maximum number of epochs for which the model is trained for one task
  "max_n_better_results": 5,             
# training stops, if validation loss has not improved for the last 'max_n_better_results' epochs
# the model with the best validation loss is returned
    "lr_decay": {"decay_rate": 0.1, "decay_every_n_epochs": 30},
}
```

## "loss"
One can scale the loss function by a constant factor e.g. 2 (```"scaling":  2```).

In the following an exemplar extract of a config is given:
```
"loss": {
  "loss_function": {"name":  "cross-entropy", "scaling":  2},
# for "supervised-learning" normally "cross-entropy" is used, "hinge-loss" is also available
# for the other learning methods various customized domain distance metrics are implemented to provide different losses (refer to `{DIR_TO_REPO}/path_learning/loss`) 
  "loss_aggregator": {"name": "mean"},
# currently "mean" is the only available loss aggregator
  "report_frequency": 100
# frequency of which the loss is reported during training (refer to `{DIR_TO_REPO}/path_learning/loss/base_loss.py`)
}
```

## "tasks"

For each task a variety of transforms can be applied to the input whereas their strength can be adjusted (e.g. ```{"transform": "blur": {"percentage": 1.0}}```). 
For more details refer to `{DIR_TO_REPO}/path_learning/dataloaders`.

Also two transforms for the labels are available: One can assign a label to each image randomly whereas the ratio of each class is preserved (```"target_transform": {"random_labels": {"state": true}}```)
or the labels of the classes can be permuted (```"target_transform": {"permute_labels": {"state": true}}```).

The learning rate can also be set for each individual task, which then overwrites the learning rate given in the optimizer for this task (```"loss": {"task_lr":  0.002}```).

The controller settings can as well be specified for each task individually. 

In the following an exemplar extract of a config is given:
```
"tasks":[
# first task:
    {"task_type": "supervised-learning", "model_to_train": "untrained",      
# next to "supervised-learning" one has also the options "supervised-grad-reg-learning", "semi-supervised-learning" and "domain-confusion"
# for more details refer to `{DIR_TO_REPO}/path_learning/learning`
# for the very first task one choses an "untrained" model
      "domains": {"target": {"transform": {"blur": {"percentage": 1.0}}}},   
# when chosing a transformation pay attention to the meaning of the value, e.g. for blur 1.0 means maximum blurring
      "dataloaders": {"labeled_dataloader": "target"},
# for "supervised-learning" the dataloader configuration is always as above
# for the other learning methods `{DIR_TO_REPO}/path_learning/dataloaders`
      "controller": {
           "max_n_epochs": 20,
           "epoch_stop": true         
# learning stops after "max_n_epochs" epochs and returns the model of the last trained epoch instead of the model with the best validation loss
           }
    },
# second task:
    {
      "task_type": "supervised-learning", "model_to_train": "prior_task",     
# the model which was trained on the first task ("prior_task") is used as inital model (this is a very important setting for our sequential learning experiments!)
      "domains": {"target": {"transform": {"blur": {"percentage": 0.0}},
      "target_transform": {"random_labels": {"state": true}}}},
      "dataloaders": {"labeled_dataloader": "target"},
      "loss": {"task_lr":  0.002}
    }
```
