We provide the .sh file for each algorithm on each tested task to facilitate readers to reproduce our results.

To profile the total time used to calculate the iteration direction of SCRN and SHSODM, one may use the profile_time.sh, and uncomment the code from Line 69 to Line 101, Line 109, Line 160, Line 161, and Line 242 to Line 249 in /garage/torch/optimizers/SCRN_optimizer.py 