# Reinforcement Learning for Durable Algorithmic Recourse

Here we provide the code for simulating a **competitive, limited-resource recourse environment** and training two reinforcement learning agents: the **recourse recommender** that learns how each candidate should modify their feature to reach a goal score, and the **predictor**, that chooses the appropriate target score. The goal is to produce recommendations that remain **feasible** and **robust** over a time horizon $T$.

---

## Running the Code

To reproduce the experiments, start by running:

```bash
python RL_agent2.py
```

This script pre-trains the recourse recommender.
Once trained, the recommender can be plugged into the predictor training by running:

```bash
python RL_agent1.py
```
In RL_agent1.py, several parameters can be set:

- `beta`: controls the difficulty of the setting.
- `t_validity`: desired duration of the recommendation.
- `alpha`, `tau`: weights for Recourse Reliability and Recourse Feasibility.
- `method`: choice of recourse recommender ("ours", "Ustun", "Wachter", "DiCE").

After training, the code outputs the average Recourse Reliability and Recourse Feasibility over 10 evaluation episodes.

---

## File Overview
- `RL_agent1_baseline.py`: for testing the standard approach of choosing the last-seen threshold as target score.
- `environments.py`: Defines the environments where the two agents are trained. The classes in this file simulate candidate behavior under limited resources. Parameters include:
    - `growth_k`: number of candidates entering the environment at each time step.
    - `threshold`: number of candidates accepted at each time step.
- `data_generation.py`: Generates synthetic datasets of candidate features, both for training the predictive model and for generating new candidates over time.
- `scorers.py`: Implements several machine learning models for predicting candidate scores.
- `recourse.py`: Provides baseline recourse methods.
- Other scripts: Contain helper functions for computing recourse methods.