<div align="center">

# RExBench : Can coding agents autonomously implement AI research extensions?
</div>

### 🎒 Dataset
: [Dataset Link](https://drive.google.com/file/d/1v4prWb-5lu69teNTdd6PZwsdMXxz_xqn/view)

### 📂 Repository Structure

```bash
.
├── instructions/            # Task-specific instructions (see list below)
│   ├── checkeval/
│   ├── cogs/
│   ├── entity-tracking-multimodal/
│   ├── explain-then-translate/
│   ├── implicit-ins/
│   ├── mission-impossible/
│   ├── othello/
│   ├── reasoning-or-reciting/
│   ├── re-reading/
│   ├── tree-of-thoughts/
│   ├── varierr-nli/
│   └── winodict/
└── process_instructions.py     # Script for processing instructions
```

Each subdirectory inside instructions/ contains an instructions.md file that describes the task setting.

### ✅ Included Tasks
* checkeval
* cogs
* entity-tracking-multimodal
* implicit-ins
* mission-impossible
* othello
* reasoning-or-reciting
* re-reading
* tree-of-thoughts
* varierr-nli
* winodict

### 🧠 Baseline Agents
* Agent 1: aider 
* Agent 2: OpenHands 
* Agent 3: Claude Code