# TRIXMED: Triage-Routed Mixture of Experts Framework for Interpretable Drug Recommendation

## Description
We introduce TRIXMED (Triage-Routed Interpretable eXpert Medicine), a novel framework that integrates Mixture of Experts (MoE) architecture with triage-inspired routing mechanisms for personalized drug recommendation. TRIXMED addresses patient heterogeneity by introducing specialized experts that handle distinct patient subgroups, while ensuring interpretability through a clustering-based routing strategy. Our approach employs a unique warm-up training phase followed by feature extraction and patient stratification, enabling transparent expert routing based on patient characteristics. 

## Installation
To set up the environment and install dependencies, run the following command:

```bash
pip install -r requirements.txt
```

## Usage
To run the main scripts, use the following commands:

```bash
sh run_training.sh
```

## Project Structure
- **camelidae/**: Contains model configurations and utilities.
- **data/**: Directory for data files.
- **llama2_dense_to_moe_sft_cl.py**: Main script for model transformation.
- **run_training.sh**: Shell script to initiate training.

## Dependencies
Key dependencies include:
- bitsandbytes
- numpy
- pandas
- peft
- scikit-learn
- sentencepiece
- torch
- tqdm
- transformers
