# Code for Beyond Cosine Similarity: Introducing the Unified semantic Similarity Metric Benchmark (USMB) for Text Similarity Measurements Paper

To reproduce our results, you must first create a `.env` file in the root directory with an `OPENAI_API_KEY` and a `COHERE_API_KEY`.

Next, install all requirements from `requirements.txt`.

Next, run `queue_file.py` in the `usmb` folder, which goes through each model and task and outputs the result to a folder labeled `data`. If you want to speed up a set of experiments, try reducing the max number of datapoints for that specific congifuration. All datasets and models will be downloaded locally from huggingface when the code is run, so be aware of this (will probably take 25-30GB of space).
