This repository contains code and data for submission LET THE RULE SPEAK: ENHANCING IN-CONTEXT LEARNING DEBIASING WITH INTERPRETABILITY.

### Pre-requisites
Make sure you have Conda installed and follow these steps to set up the environment:

	conda install -n FuRud python=3.10
	conda activate FuRud
	pip install -r requirements.txt


### Data Preparation
1. To obtain ICL outputs, we follow (Lin and You, 2024; https://arxiv.org/pdf/2405.07623):

(1) Prompts for main experiments use label token options, and a task instruction like:

Please classify the news articles into the categories of World, Sports, Business, and Technology.

(2) Prompts for letter based experiments use letter options, and a task instruction like:

Please classify the news articles into the following categories. Options: A. World B. Sports C. Business D. Technology

The ICL probability vectors used in the work will also be released with the source code upon paper publication. The vector files are submitted along with the paper, and are put under vectors/llama2-13b.


### FuRud Optimization Example
1. FuRud optmization parameters are located at 

	config/default_params.json 

You may change the parameters other than the default settings

2. An example script for running FuRud is located at

	scripts/example.sh,

which contains the following command:

	python furud-main.py \
			-c config/default_params.json \
			--vec_dir vectors/llama2-13b/{ds}_llama2-13b_shot1_seed{sd}

where ds is an evaluation benchmark dataset name and sd is in [0,1,2] used as a random seed when randomly selecting the 1-shot demonstration. Shot1 refer to using a single shot for ICL. The {ds}_seed{sd} directory contains the vector files obtained following data preparation (or use the vector files submitted along this paper), including opt.txt and test.txt. Run it from directory ./furud/model. Run it from directory ./furud/model.

