### The synthetic experiments

For the synthetic experiments, the codes are in the folder `/Synthetic`,
the results can be obtained by running 
`python normalizer.py`.
The final training MSE and test MSE are printed in the terminal.
The figure is saved in `./result.png` 


### The LRA benchmark

For the LRA benchmark, we follow the framework from this official [S4 model repo](https://github.com/HazyResearch/state-spaces/).
We only change the training scripts and the model scripts to get the generalization measure.
Specifically, for the initialization scheme, the rescale operation is defined in the `_shared_step` function in `train.py`.
If the initialization scheme is not applied, then comment from Line 324 to Line 330.
The regularization coefficient `lam` is defined in Line 334.
If no regularization is applied, then set `lam=0`.

After processing the data according to the [S4 model repo](https://github.com/HazyResearch/state-spaces/), the training commands are as follows:
```
python -m train experiment=lra/old/s4-lra-listops wandb=null
python -m train experiment=lra/old/s4-lra-imdb wandb=null
python -m train experiment=lra/old/s4-lra-cifar wandb=null
python -m train experiment=lra/old/s4-lra-aan wandb=null
python -m train experiment=lra/old/s4-lra-pathfinder wandb=null
```




