main.py is the main python 3 file.

Training configuration (requirements_all.txt; Python 3.8.2 was used in the experiments), the most important packages:
numpy==1.24.4
scikit_learn==1.3.2
torch==2.1.0+cu118
torchvision==0.16.0+cu118
torchtext==0.16.0


Inside main.py you can set:
method=20 #gradient-based training
method=21 #AG-1 variant
method=27 #AG-2 variant
method=37 #AG-3 variant
method=30 #AG-3-1 variant

To run "Linear" variant, then comment "from algorithms import *" and uncomment "from algorithms_linear import *"

dataset='mnist','fashion_mnist', 'imdb', or 'imagenet_ood'
optimizer_type=SOAP, optim.RMSprop, or optim.Adam
model_nr=6: Model A, 9: Model B, 12: Model C, 13 ResNet-152, 14 ResNet-152-GELU, 15 ResNet-152-SiLU, 18 ResNet-152-Sig
trainings_same_model: number of trainings per particular hyperparameters
trainings_different_model: number of different hyperparameter sets


Imagenet-OOD data is needed to run ResNet-152 trainings. The dataset can be downloaded from the official website of Imagenet
glove.6B.50d.txt is needed to run IMDb training runs, which can be downloaded e.g. from https://www.kaggle.com/datasets/watts2/glove6b50dtxt