Testing Recurrent vs Non-Recurrent Outlier Detection Algorithms:

This was tested and run on macOS Big Sur. There are some known issues with Windows, such as the downloading and processing of the SWAN-SF dataset.

Installing TODS and dependencies:

Please follow these instructions for installing of the TODS outlier detection package:

For this research, we require a forked version of the TODS code. However, since the reviews are anonymised I cannot provide a link to the forked code. Once I am able to, I will replace the code block below with a link to download and install the forked TODS code.

For now, the below TODS installation will not install the forked version and so the pipelines will not run.

!pip install -e git+https://github.com/datamllab/tods.git#egg=tods

Please follow the link to https://zenodo.org/record/3884398#.YVyEOy8w1qs and download 1_gecco2018_water_quality.csv.

Then place the file in the directory ./src/tods/benchmark/realworld_data/data/script/raw_data/

Finally, run the following code:

%cd ./src/tods/benchmark/realworld_data/data/script/
%run -i ./gecco.py
%cd ../../../../../../

The following code block will not run on windows because the file names in the folder are incompatible:

%cd ./src/tods/benchmark/realworld_data/data/script/
%run -i ./swan_sf.py
%cd ../../../../../../

Installing CRTOD and dependencies:

!pip install -e crtod_code

Now, please restart the IPython kernel before running the below:

Importing torch:

Please choose whether to run on cpu or another device. This only affects the transformer models which are built on pytorch. The TODS built models will always run on the CPU.

import torch
device = torch.device('cpu') # ['cpu', 'cuda']
device

device(type='cpu')

Synthetic Data:

path_to_TODS = './src/'

%run -i ./synthetic_runner.py

%run -i ./synthetic_loader.py

Real-World Data:

%run -i ./realworld_runner.py

%run -i ./realworld_loader.py