# Residual Scheduling: A New Reinforcement Learning Approach to Solving Job Shop Scheduling Problem

## Files for datasets and pre-trained weights

All benchmarks and data are in the 'datasets' folder, and are listed as follows:
- datasets
	- JSP (inclulding ta, abz, la, orb, ft, swv, yn, JSP_validation and Random_generated)  
		ta is TA benchmark  
		abz is ABZ benchmark  
		la is LA benchmark  
		orb is ORB benchmark  
		ft is FT benchmark  
		swv is SWV benchmark  
		yn is YN benchmark  
		JSP_validation contains the instances for validation   
		Random_generated contains the instances for the experiments in "Discussions" section  
	- FJSP (including Brandimarte_Data, Hurink_Data and Song_Data)
		Brandimarte_Data is known as MK benchmark
		Hurink_Data is known as LA benchmark
		Song_Data is from the work "Flexible Job-Shop Scheduling via Graph Neural Network and Deep Reinforcement Learning"

All pre-trained weights are in the 'weight' folder, and are listed as follows:
- weight 
	- RS_JSP/best
	- RS_FJSP/best
	- RS+op_JSP/best
	- RS+op_FJSP/best

## Installation

Setup the virtual environment.
```c
podman run --rm -it -v $PWD/ResidualScheduling:/ResidualScheduling -w /ResidualScheduling pytorch/pytorch:1.11.0-cuda11.3-cudnn8-runtime
```

Install required packages in the environment.
```c
pip install torch-geometric opencv-python plotly matplotlib gym tensorboard pandas colorhash
```
## Run training
Follow the example to run a FJSP training procedure(RS). And there are some parameters for ablation studying.
```c
python3 train.py --date=train --instance_type=FJSP --data_size=10 --delete_node=true
```

## Run validation (be sure that there are some weights to be validated)
```c
python3 valid.py --date=train --instance_type=FJSP --delete_node=true --valid_dir='./datasets/FJSP/Song_Data/data_dev/1510'
```
It will valid all weights in './weight/train'

## Reproduced the result in paper
Follow the example to run a FJSP testing 
```c
python3 test.py --date=test --instance_type=FJSP --delete_node=true --test_dir='./datasets/FJSP/Hurink_Data/vdata' --load_weight='./weight/RS_FJSP/best'
```
Follow the example to run a FJSP testing (RS+op)
```c
python3 test.py --date=test --instance_type=FJSP --test_dir='./datasets/FJSP/Hurink_Data/vdata' --load_weight='./weight/RS+op_FJSP/best'
```

### Similarly, for JSP
Follow the example to run a JSP testing (RS)
```c
python3 test.py --date=test --instance_type=JSP --delete_node=true --test_dir='./datasets/JSP/ta' --load_weight='./weight/RS_JSP/best'
```
Follow the example to run a JSP testing (RS+op)
```c
python3 test.py --date=test --instance_type=JSP --test_dir='./datasets/JSP/ta' --load_weight='./weight/RS+op_JSP/best'
```

## Hyperparameters list
```c
    python3 train.py \
    --device='cuda' \
    --instance_type='FJSP' \
    --data_size=10 \
    --max_process_time=100 \
    --delete_node=False \
    --entropy_coef=1e-2 \
    --episode=1000001 \
    --lr=1e-4 \
    --step_size=1000 \
    --hidden_dim=256 \
    --GNN_num_layers=3 \
    --policy_num_layers=2 \
    --date='Dummy' \
    --detail=None \
    --test_dir='./datasets/FJSP/Brandimarte_Data' \
    --valid_dir='./datasets/FJSP/Song_Data/data_dev/1510' \
    --load_weight='./weight/RS_FJSP/best'
```
instance_type is either "JSP" or "FJSP".  
data_size needs to be larger than 3.  
max_process_time needs to be larger than 0.  
delete_node is either "Ture" or "False": "True": Remove completed operations; "False": Keep completed operations.  
entropy_coef is entropy coefficient, and suggested to be a value in [0, 1].  
lr is learning rate.  
step_size is the steps to decay the learning rate.  
hidden_dim is the hidden dimensions in network layers.  
GNN_num_layers is the layers for GNN.  
policy_num_layers is the layers in the policy network.  
date is for the name to record the results.  