# Domain Evaluation Pipelines

This contains a properly structured Domain Evaluation pipeline with the directory structure arranged as below:


``` bash 
.
├── data/
│   ├── cpp.json
│   ├── python.json
│   ├── finance.json
│   ├── medical.json
│   ├── math.json
│   └── science.json
│
├── output/
│   ├── stage_1_sweep_results/    # Raw outputs from the initial component-wise sweep.
│   ├── stage_2_generated_configs/ # Auto-generated Python configs from the scout script.
│   └── stage_3_targeted_results/ # Final results from the targeted fine-tuning runs.
│
├── 01_component_sweep.py         # Stage 1: Runs broad fine-tuning on MLP, Attention, or both.
├── 02_scout_and_architect.py     # Stage 2: Analyzes Stage 1 results and creates experiment configs.
├── 03_targeted_finetune.py       # Stage 3: Runs targeted, layer-specific fine-tuning experiments.
│
├── README.md                     # This documentation file.
└── requirements.txt              # A list of all required Python packages.
```


---

## Installation

To run the codes, please download the requirements using:

```bash
pip install -r requirements.txt
```

## Running the Codes

- Before running, you need to set all required inputs using `os.environ.get` in the starting part of the respective code.  
- Results will be stored in the specified **LOG_dir** provided as input.  
- Use the hyperparameters as mentioned in the paper to reproduce the results.  


