# Main accuracy/perplexity experiment

This experiment contains scripts to assess different numerical formats

## How to use

First you need to set-up the config of the experiment. Check `experiment_config.py` for more info
Then, `run.sh` will run the experiments and then plot the results of them.

## OWQ reproduction

Before running the experiment for the OWQ numerical format, you need to generate the quantized model using their script.
You can do that by going to `src/transformers/owq/scripts` and running `opt_end_to_end_evaluation.sh`. For example, with parameters `0 1.3b` you will get 1.3b model.
Then you will need to place this model to the `intermediate/owq-paper` folder of this experiment (`src/experiments/k-bits-reproduction/intermediate/owq-paper`), you can create
it manually if it doesn't exist.