We provide the code to run our instruction tuning experiment.
-Hardware: a single A100 GPU with 40G/80G memory
-Dependencies:
	1. Follow the installation in https://github.com/princeton-nlp/LESS (This is NOT our repo.)
	2. Additionally, install the following:
		pip install datasets
		pip install wandb
		pip install huggingface_hub
		pip install vllm
		pip install evaluate>=0.4.0 
		pip install openai>=1.5.0
		pip install faiss-cpu
-Prequisites
	1. Download the data from https://huggingface.co/datasets/princeton-nlp/less_data (This is NOT our repo either), put them into a folder named "data"
	2. Follow https://github.com/princeton-nlp/LESS to encode the examples and customized GRADIENT_PATH and VALIDATION_GRADIENT_PATH in code/kdeknn/prob_assign.sh
	3. Follow https://github.com/princeton-nlp/LESS to select data using their method.
-Run the experiments
	cd code
	bash run.sh
