# Countdown code and datasets

## Datasets
We provide the generated datasets. Each file has 100 entries per size

 * [CD](datasets/CD_4_50.jsonl.gz) our generated data, sizes 4 to 50  
 * [RG](datasets/RG_4_50.jsonl.gz) Reasoning Gym, sizes 4 to 50 
 * [SoS](datasets/SoS_4_9.jsonl.gz) Stream of Search, sizes 4 to 9

We also provide the [PDDL domain](PDDL/) for the CD dataset


## Installation

Use the package manager pip to install the required dependencies.

```bash
pip install -r requirements.txt
```

## Data generation

### Our benchmark (CD)
To generate data for state size 4:
```bash
python dataset_generator.py --state-size 4 --num-iterations 100000
```


### Reasoning Gym (RG)
To generate data for state size 4:
```bash
python dataset_generator.py --reasoning-gym --state-size 4 --num-iterations 100000
```

### Stream of Search (SoS)
To generate the data, we used the code from the original repository with a few minor changes. To reconstruct it:
```bash
# Cloning the repo
git clone git@github.com:kanishkg/stream-of-search.git
cd stream-of-search.git 
# Apply the changes to the code
git apply ../sos_diff.patch
# Install required libraries
pip install tiktoken tqdm
# Run the script
./scripts/gen_task.sh
```
The data will be create in `src/data/`

## PDDL generation

To generate PDDL data for the SoS dataset:
```bash
mkdir sos_pddl
python pddl_generator.py datasets/SoS_4_9.jsonl.gz sos_pddl
cp domain.pddl sos_pddl/
```

## AutoToS generated code
The file `autotos_functions.py` includes the code extracted from AutoToS LLMs outputs for the three tested models, together with the code of the DFS algorithm used, as well as the validation code for the search output. The ids for the functions by the source model are:
```python
deepseek = ["20250509_154448","20250509_164317","20250509_183049","20250509_160231","20250509_171121"]
llama = ["20250512_161832", "20250512_165326", "20250512_171238", "20250512_171658", "20250512_172254"]
qwen = ["20250512_175943", "20250512_183704", "20250512_183926", "20250512_191957", "20250512_193400"]
```

To run it on an example problem (e.g., [1,2,3,4] with the target 12) do
```bash
python autotos_functions.py --state "1 2 3 4" --target 16 --code-id 20250509_154448
```

